500,000 requests/sec – Modern HTTP servers are fast

A modern HTTP server running on somewhat recent hardware is capable of servicing a huge number of requests with very low latency. Here’s a plot showing requests per second vs. number of concurrent connections for the default index.html page included with nginx 1.0.14.

With this particular hardware & software combination the server quickly reaches over 500,000 requests/sec and sustains that with gradually increasing latency. Even at 1,000 concurrent connections, each requesting the page as quickly as possible, latency is only around 1.5ms.

The plot shows the average requests/sec and per-request latency of 3 runs of wrk -t 10 -c N -r 10m http://localhost:8080/index.html where N = number of connections. The load generator is wrk, a scalable HTTP benchmarking tool.

Software

The OS is Ubuntu 11.10 running Linux 3.0.0-16-generic #29-Ubuntu SMP Tue Feb 14 12:48:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux. The following kernel parameters were changed to increase the number of ephemeral ports, reduce TIME_WAIT, increase the allowed listen backlog, and the number of connections Netfilter can track:

  echo "2048 64512" > /proc/sys/net/ipv4/ip_local_port_range
  echo "1" > /proc/sys/net/ipv4/tcp_tw_recycle
  echo "1" > /proc/sys/net/ipv4/tcp_tw_reuse
  echo "10" > /proc/sys/net/ipv4/tcp_fin_timeout

  echo "65536" > /proc/sys/net/core/somaxconn
  echo "65536" > /proc/sys/net/ipv4/tcp_max_syn_backlog  

  echo "262144" > /proc/sys/net/netfilter/nf_conntrack_max

The HTTP server is nginx 1.0.14 built with ./configure && make, and run in-place with objs/nginx -p . -c nginx.conf.

nginx.conf

 
worker_processes     16;
worker_rlimit_nofile 262144;

daemon off;

events {
  use epoll;
  worker_connections 16384;
}

error_log error.log;
pid /dev/null;

http {
  sendfile   on;
  tcp_nopush on;

  keepalive_requests 100;

  open_file_cache max=100;

  gzip            off;
  gzip_min_length 1024;

  access_log off;

  server {
    listen *:8080 backlog=16384;

    location / {
      root   html;
      index  index.html;
    }
  }
}

Hardware

A dual Intel Xeon X5670 with 24GB of RAM from SoftLayer. The X5670 has 6 cores @ 2.93 GHz, 2 threads per core, /proc/cpuinfo shows 24 CPUs.