The Value of Background Fetch →
Great post by Supertop, makers of the Castro podcast app:
The background fetch API is a game-changer for iOS developers. It has the potential to free us of significant server and infrastructure overheads. This is particularly relevant at a time when many developers are wondering how to stay independent. For Castro, the decision was an easy one and we strongly advocate that other developers take full advantage of this new API as well.
(Since I’m working on a competitor to Castro, take all of my comments here with a grain of salt.)
Service-backed apps still have a lot of advantages and exclusive capabilities over iOS 7’s Background Fetch. I think server-side crawling is still the best choice for podcast apps and feed readers, for which users want fast updates to collections of infrequently updated feeds.
Overcast has been crawling tens of thousands of podcast feeds every few minutes for the last 6 months using standard HTTP caching headers. In the last week, 62% of all requests have returned 304 (“Not Modified”). Many of the rest returned the entire “new” feed, but none of the episodes had actually changed, making the server download and process hundreds of kilobytes unnecessarily.
An app using Background Fetch needs to make all of those fruitless requests just to get the handful of occasional changes. All of those requests cost processor time, memory, battery power, and data transfer. And each copy of the app needs to download those hundreds of wasted kilobytes when a server erroneously reports an unchanged feed as new.
While a server can easily crawl a feed every few minutes without problems for either side, 20,000 copies of an app polling a show’s feed will be noticeable — and they won’t even get the updates as quickly as the server-side crawler since they’re running less frequently.
A server can simply send a silent push notification to all subscribed apps when there’s new data in a feed, and each app can download just the changes. With infrequently updated feeds, like podcasts, this leads to huge savings in battery life and transferred data over time.
I agree with Supertop that it’s important to minimize hosting costs, especially as App Store economics become more challenging, but hosting is cheaper than ever with SSDs and modern CPUs. At launch, I’ll be paying more each month for my health insurance than all of the web-hosting expenses, and I bet this will remain the case for a while.
But getting hosting this cheap requires a bit more work. You probably can’t use most cloud services, you probably need to learn basic Linux server management (it’s not as hard as you think), and you need to be a bit careful with your decisions and implementation. I’m still using VPSes, dedicated servers, PHP, and MySQL because I know how to host scalable web services very cheaply this way.1
Background Fetch is a great option, but there’s still a lot of value in a web service if you can make one.
-
The entire Overcast feed-crawling infrastructure can run on a $40/month Linode VPS. And Linode hasn’t even deployed SSDs yet.
To give myself more headroom for testing and the launch, whenever that actually happens (not soon), I recently switched to a dedicated server at Limestone Networks (that’s a referral link) with a quad-core Xeon E3-1270 v3, two 100 GB Intel enterprise SSDs in RAID-1, 16 GB of RAM, and far more bandwidth than I can use for $277/month.
I still can’t believe it’s that cheap. This could have been Instapaper’s master database server, which cost $1,200/month just a few years ago. I don’t think there’s a better value in dedicated servers these days than a Xeon E3 with an SSD or two. ↩︎