The great APC issue
APC was actually showing us two significant problems:
- We were using it as a general data cache, like Memcache. It’s MUCH faster than Memcache when working properly because it’s process-local to PHP. But it seems to have some sort of resource leak (never pinpointed it). Over time, Apache processes would bloat and generally slow down until they became unresponsive, maxing out the CPUs (but not disks, and not enough RAM to be fatal). This was particularly painful to debug because we assumed our v3 code was just being incredibly slow somewhere, so we spent most of our time profiling and optimizing it, but the problem kept happening. (Fortunately, we get to keep all of those optimizations, so it wasn’t a waste.)
- Like eAccelerator, MMCache, and XCache, APC also does PHP compiled-opcode caching. It did this well enough, but over time, an otherwise healthy and fast server would have its httpd child processes start segfaulting. Over a few minutes, the segfaults would cascade to all child PIDs and kill Apache, requiring a hard httpd restart (not a graceful).
So I switched to eAccelerator. We had problems with its PHP 5.2 compatibility a while ago, but they seem to have fixed that. So far, it’s fine, and neither of those problems have cropped up again.