Marco.org

I’m : a programmer, writer, podcaster, geek, and coffee enthusiast.

Web Hosting For App Developers

Justin Williams captures much of the complexity facing modern app developers in The Parts of Your Platform:

Ignoring the cloud or web services because they are out of your comfort zone is no longer an option.

But we need to sit down and have a talk about web hosting. Seeing well-meaning developers get burned, abandoned, or ripped off by unexpected changes, limitations, and costs in the cloud-service landscape is too painful for me to watch in silence.

The common wisdom, which Justin suggests, is to go directly to a highly abstracted, proprietary cloud service or a higher-level hosted back-end — the kind that are so high in the clouds that they call themselves “solutions”. But the “BaaS” landscape is still very unstable with frequent acquisitions and shutdowns likely, and hosting on VPS-plus-proprietary-services clouds like Amazon Web Services or higher-level services like Heroku or App Engine can get prohibitively expensive very quickly.1 Developers who build everything on these services by default would probably be shocked at how cheaply and easily they could run on dedicated servers or unmanaged VPSes.

Running your own servers really isn’t hard. Most developers reject the idea outright without even trying because it’s unfamiliar and intimidating. It’s considered an extreme, horrible, unfathomable situation that must be avoided at all costs, usually by people who have never tried it.

But that’s a fallacy. There’s a learning curve and necessary integration work for every back-end option, from iCloud and Dropbox to your own colocated servers. AWS, Azure, Heroku, App Engine, Parse, and similar services aren’t free, easy, or automatic. (Neither is “scaling” with them, regardless of what you’ve heard.) Hosted infrastructure is like sync: it has a minimum, unavoidable level of complexity to accommodate. You can’t just check a box or set a BOOL and have it all taken care of for you.

If you’re going to invest your time into learning any of these options, you might as well learn the timeless one that’s most likely to survive every acquisition, language, trend, and paradigm shift: basic Linux server administration.2

Wait! Don’t leave! Hang in there.

Modern Linux server administration is much easier than you think. If you can write a halfway decent app, you can manage a Linux VPS in your sleep.

You don’t need to compile kernels, build anything from source code, partition any disks, or deal with iptables in most cases. The defaults of good distributions and packages are almost always very secure. And once you set everything up, you can leave it running largely untouched indefinitely. You’ll probably never be woken up at 3 AM to reboot anything or delete log files.

Take the weekend to open a Linode account, create the cheapest instance ($20/month, billed hourly), and plow through initial setup of something substantial, like your own basic Ruby, Python, PHP, or Node app with a boring old database (MySQL or Postgres) and webserver (nginx or Apache) if necessary.3 If you mess up so badly that Google can’t help you, just delete or restore the instance and start over. Cheap VPSes have made it easier than ever to experiment and learn.

Want a scalable setup? Make three ($60/month). Two are your webservers and one is your database.4 Put a virtual load balancer in front of the two webservers ($20/month).5 At any time, you can then easily take either of the webservers down for a few minutes to upgrade to a higher-powered instance without taking the whole service down. If you get an influx of traffic one day, just clone a few more webservers behind the load balancer. After the load subsides, delete the ones you don’t need anymore.

Self-managed VPSes are the best option today for most developers hosting web apps. Good VPS services provide many of the benefits and easier scaling of higher-level services, like easy cloning, easy backups, fast upgrades and downgrades, and elimination of most tedious hardware management.

If you end up getting so big that you need more horsepower than VPSes can practically or affordably offer, you can switch from standard Linux VPSes to cheaper, faster dedicated or colocated servers very easily using most of the skills and tools you already have. You can even mix and match VPSes and dedicated servers at some hosts.

If any particular host starts going downhill or is no longer price-competitive, you can take all of your skills, tools, and infrastructure and move them to another host. Unlike proprietary cloud platforms, regular VPSes and servers are a competitive commodity market. Linode sucks? Try DigitalOcean, which offers effectively the same product at similar prices. Rackspace too expensive? Try SoftLayer or Limestone Networks. Simply avoid all hosts’ proprietary, hard-to-replace products and stick to their standard VPSes and servers, and your service and knowledge become portable and resilient.6

Tumblr taught me by necessity, especially in the early days when we had no staff and very little money, how to develop and host a high-traffic web service cheaply, easily, and sanely. I couldn’t let the servers require too much maintenance because server administration wasn’t my main job — we didn’t have a dedicated server administrator for the first few years, so I just had to set things up such that they didn’t need much administration.

In 2006, that was a rarely-needed specialty. Even when I launched Instapaper in 2008, hardly anyone making consumer apps needed those skills. But if I hadn’t learned them at Tumblr, Instapaper could never have grown independently, The Magazine wouldn’t exist, and Overcast would be much more limited.

Today, the ability to sanely and economically run a web back-end is a basic requirement for a lot more developers, and it’s only going to become more necessary in the future. Many apps are already expected to have sync, connectivity, and web features, and those expectations will only spread to more categories in the future. Refusing to embrace this shift will severely limit your apps and may cost you your business.

Start building those skills now.

  1. “Cloud” is such a buzzword these days because cloud hosts, continuing the web-hosting tradition, have very fat margins, low barriers to entry, and high barriers for customers to leave.

    Virtualized cloud services are web hosts’ dream: there’s much more opportunity for branding and hand-wavy products with ambiguous benefits and unverifiable claims, less focus on numbers that could be comparison-shopped (what, exactly, is “4 CPU”?), highly profitable usage of old hardware (often running an undifferentiated mix of hardware up to 5 years old or more), and more room for proprietary lock-in. And they can still price their services as if RAM was expensive: the biggest con in the history of web hosting. It’s high-profit wins all around. ↩︎

  2. Linux distributions are an unfortunate oversupply of paralyzing choices. The easiest path is to learn one major distribution very well and use it everywhere. You want conservative, slow-moving, and very popular: that way, updates almost never break anything and it’s all very stable (conservative), you don’t need to re-learn the basics and tools constantly (slow-moving), and there are tons of Google results and tutorials for every question you’ll have (very popular). I chose CentOS in 2005 with no regrets: it’s basically a free version of Red Hat Enterprise Linux that exceptionally satisfies all three conditions. I’ve heard Debian is also a solid choice. Always deploy the latest official (stable) 64-bit release. ↩︎

  3. Boring old is the key to server-administration happiness. Stick to the boring and the old, and you’ll rarely need to deal with anything. The lower down the stack, the more important that becomes.

    This is why I still very happily use MySQL (InnoDB-only) instead of a trendier, newer database: it’s very fast when used properly, and I’ve never seen it crash, corrupt data, or perform irregularly. And I’ve run a lot of heavily-used MySQL servers. Not a single crash, ever. The only other heavily used server tool I can say that about is HAProxy. ↩︎

  4. Don’t forget to schedule automatic database backups, since the host’s backups probably won’t be consistent for the database files. For MySQL, mysqldump is fine until your database gets huge, at which point you should consider xtrabackup.

    My favorite option: make a replicating slave database server (another $20/month) whose sole responsibility is backups. Every day, have it shut down MySQL, tar-gz the entire /var/lib/mysql data directory with the binlog number and position in the filename, encrypt it, upload it to a S3 account dedicated to this purpose, and restart MySQL. This is by far the most reliable and fastest-to-recover MySQL backup method that I’ve ever used.

    (Bonus points: between backups, every few minutes, rsync the binlogs somewhere. Then you have point-in-time, query-level recovery to any point up to a few minutes ago.) ↩︎

  5. If a virtual load balancer isn’t available at your host or you don’t want to use one, just use a software one, like the excellent HAProxy, running on the cheapest VPS available. (They’re very low-needs.) ↩︎

  6. I also maintain a giant shell script that configures everything I need on a clean install of CentOS. Once you’re comfortable with the basics, I suggest doing this, as it makes it easier to set up new servers or switch hosts.

    Making such a script is much easier with VPSes, where you can start a new one, test it out, change it as needed, delete the instance, and try again on a clean one.

    Ideally, your servers should be disposable and easily recreated. The only backups you should need are your source code (which should include any required server-setup scripts) and your database’s data. ↩︎

◆