Re: HAProxy, NGINX and Rails anyone?

On Thu, Jul 31, 2008 at 03:20:09AM -0400, Brian Gupta wrote:
> That said, I'm surprised that they selected nginx for load
> balancing, as they were only able to push 1.2GB of traffic through
> nginx. Haproxy has been tested all the way to handle a full 10GB of
> traffic through a single server.

Be very careful Brian, I think you're mixing two things. It's probable that wordpress tested on real-world traffic, while my benchmark was meant to find the absolute maximum performance. Real-world benchmarks are used to estimate the ability to do the job properly. Component benchmarks are used to find the bounds beyond which you're sure you will never go.

Now, that said, I know about people who have a handful of machines all saturating gig pipes all the day with moderate CPU usage. And judging by the workload (large transfers essentially, no problem with running one process per CPU core), it should be possible to use a 10 Gig NIC in one single machine. But this case is extreme, as most people serve smaller files. From my first series of tests, you need to serve files larger than 50kB in average to go beyond 5 Gbps (that was about 13000 hits/s). And you needed an average of 500kB to reach 9 Gbps.

You should also be aware that reaching these loads on one machine requires a lot of system tweaking. You can run out of memory in seconds, and if the system runs low on network buffers, you observe terrible performance. It's always a trade between raw performance and reliability.

These days, I would say that 10Gig becomes affordable but a lot of work is required to get the best of it, while reaching gig speeds is almost a child's game.

Regards,
Willy Received on 2008/08/01 00:04