Re: HAProxy, NGINX and Rails anyone?

On Thu, Jul 31, 2008 at 6:04 PM, Willy Tarreau <w#1wt.eu> wrote:

> On Thu, Jul 31, 2008 at 03:20:09AM -0400, Brian Gupta wrote:
> > That said, I'm surprised that they selected nginx for load
> > balancing, as they were only able to push 1.2GB of traffic through
> > nginx. Haproxy has been tested all the way to handle a full 10GB of
> > traffic through a single server.
>
> Be very careful Brian, I think you're mixing two things. It's probable
> that wordpress tested on real-world traffic, while my benchmark was
> meant to find the absolute maximum performance. Real-world benchmarks
> are used to estimate the ability to do the job properly. Component
> benchmarks are used to find the bounds beyond which you're sure you
> will never go.

Understood. I was wondering if they had haproxy properly configured. Frankly haproxy's docs are really hard to digest for newcomers, and are prolly the biggest impediment to growing the user base. It's one giant text file so you basically have to know what you are looking for already. (It's documentation geared toward advanced haproxy users and haproxy developers.) When things slow down a bit, I plan to help out with the Wiki efforts to address newbie documentation.

Now, that said, I know about people who have a handful of machines
> all saturating gig pipes all the day with moderate CPU usage. And
> judging by the workload (large transfers essentially, no problem
> with running one process per CPU core), it should be possible to
> use a 10 Gig NIC in one single machine. But this case is extreme,
> as most people serve smaller files. From my first series of tests,
> you need to serve files larger than 50kB in average to go beyond
> 5 Gbps (that was about 13000 hits/s). And you needed an average of
> 500kB to reach 9 Gbps.

I thought you had posted that you saturated a 10G NIC somewhere? (Maybe I misunderstood/misremembered something.)

> You should also be aware that reaching these loads on one machine
> requires a lot of system tweaking. You can run out of memory in
> seconds, and if the system runs low on network buffers, you observe
> terrible performance. It's always a trade between raw performance
> and reliability.

I got ya... I still say it would be interesting to see if we could improve those Wordpress testing numbers using haproxy. (Even with real world traffic it should be possible to beat 1.2Gbit/sec)

> These days, I would say that 10Gig becomes affordable but a lot
> of work is required to get the best of it, while reaching gig
> speeds is almost a child's game.

Key word, "almost". ;)

> Regards,
> Willy
>

Cheers,
Brian

-- 
- Brian Gupta

http://opensolaris.org/os/project/nycosug/

http://www.genunix.org/wiki/index.php/OpenSolaris_New_User_FAQ