Close the door when she's out (improve your web server performance)

Imagine you drive the traffic of an airport. This airport has 5 landing/runway strips that can be used simultaneously. To avoid turbulences and problems, a clearance time of 3 minutes must be wait after an airplane take off and before the next one can take off. If 2 minutes are needed for a plane to take off, it means that a maximum of 1 plane every 5 minutes can take off for a given strip. It gives us a maximum of 60 airplanes taking off every hour as a maximum for our hypothetical airport.

If we could reduce the clearance time from 3 minutes to 1, then the capacity of the airport will be increased from 60 flights per hour to 100 flights per our, that it's near a 67% gain.

Same thing happens with web servers. Most web servers allows you to configure the timeout for client connections (maximum time of inactivity before closing a connection with a client machine). Having a high connection timeout reduces your load capacity because you are losing 1 connection capacity for every connection that is just wasting time until it reaches the configured timeout value. Of course it wouldn't happen if your clients were polite and tell you when they are done. But the reality is that many times clients don't close connections (you can check it with the help of Apache logging capabilities, see my previous post Logging Apache's *_cache modules efficiency). That causes a lot of resource wasting, thus reducing your effective load capacity. Increasing the concurrency over your physical capacity will be similar to taking off an airplane without waiting for the clearance time, in the airport example, causing system instability.

Some months ago we did a performance audit for a big company that was having performance problems with their online publishing platform. In the audit we saw that their front-end web servers (remember, a front-end server must be fast always) were always at their maximum capacity of about 800 sessions per server on average, all the time (measured every 5 minutes and taking the 95 percentile). Although one of their sysadmins thought that timeouts had no relationship with performance or load, the reality was that lowering the innactivity timeout from 120 seconds to 20 seconds lowered the average of connections to 600 per server. It means:

• the servers could handle about 200 connections more at any time


• the servers were consuming about 1GB less RAM each one, so the system has more RAM for buffers, caches and so, greatly improving the performance

The conclusion is that if all the clients tell you when they are done, you will gain a lot of capacity. But if they do not (likely), then shut down idle connections quickly so another airplane can take off.