500,000 Requests/Sec? Piffle! 1,000,000 Is Better

Modern HTTP servers are capable of handling 500k requests/sec on commodity hardware. However that article ignored HTTP pipelining which can have a significant impact on performance. Buggy legacy servers and proxies prevent most browsers from enabling pipelining by default, but that is the high-latency past, not the future. In fact we can nearly double the performance of nginx 1.0.14 by enabling pipelining:

Getting to 1M requests/sec required minor tweaking of the client and server compared to the original article. nginx’s worker_processes was reduced from 16 to 14, wrk’s threads were increased from 10 to 11, and 30M requests were made instead of 10M. Maximum performance was reached with a pipeline depth of 8 and 1,100 concurrent connections.

wrk -t 11 -c N -r 30m --pipeline 8 http://localhost:8080/index.html


When pipelining is enabled wrk counts latency as the time from the first request to the last response. In this particular environment less than 400 concurrent connections result in worse latency and throughput with 8 pipelined requests, however with 400+ connections latency and throughput are significantly better.

A SPDY Future

What makes these results even more interesting is the accelerating acceptance of SPDY as a replacement for HTTP 1.1. Chrome has supported SPDY for a while, Firefox and nginx will soon, and there is an experimental Apache module. For dynamic content Jetty and Netty support SPDY today, and Netty’s implementation is already in production use by Twitter.

Persistent connections and support for multiple concurrent requests are inherent to SPDY, so a move to SPDY will reduce the number and frequency of new connections, as well as the latency caused by non-pipelined request/response cycles. This will allow HTTP servers to get closer to their theoretical maximum performance, which looks to be very high indeed for nginx.