Re: High-volume HAProxy deployment tips?

From: Krzysztof Oledzki <ole#ans.pl>
Date: Fri, 19 Dec 2008 08:29:00 +0100 (CET)

On Fri, 19 Dec 2008, Willy Tarreau wrote:

> On Fri, Dec 19, 2008 at 12:13:18AM +0100, Benoit wrote:

>> Willy Tarreau a écrit :
>>>> As for us, we had to double the haproxy boxes to get past the 40Mbps of
>>>> http traffic ....
>>>>
>>>
>>> what type of traffic do you have and what type of network cards do you
>>> have ?
>>> The smallest machines I have (fanless AMD Geode 500 MHz, 2.5W of average
>>> power
>>> consumption) already supports 100 Mbps and up to 2000 connections/s with a
>>> dirt cheap VIA Rhine NIC. So there's definitely a problem with your setup !
>>>
>>
>> Server is an HP blade, with 4 Broadcom Corporation NetXtreme II BCM5708S
>> port, bonded two by two.
>> and it's a bi dual core Xeon 5150#2.66Ghz cpu with 4Gb of memory system
>> dedicated to haproxy.
>>
>> The cpu load is low, very very low, however when at 40 or 50Mbps
>> connection time start to go up to the roof
>> ( up to 3/4s sometimes, instead of 180ms of total processing time), and
>> activating the backup server fixed this
>> so it isn't of problem with the backends

>
> OK I understand better now. At a customer's, we've been using HP blades
> with BNX2 NICs too, and we disabled them. The packet loss is terrible on
> those crappy NICs.

I can't confirm this. I have two BCM5708 NICs currently running on 2.6.26.8:

# lspci|grep Eth
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12) 07:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)

I have never experienced performance issues with these NICs and from my POV bnx2/tg3 NICs are much better than for example e1000 ones as there is no IPMI+VLAN problem with Broadcoms.

Maybe this is a firmware issue? Have you tried upgrading it? Mine is 4.4.1:

# ethtool -i eth0
driver: bnx2
version: 1.7.5
firmware-version: 4.4.1 ipms 1.6.0
bus-info: 0000:03:00.0

AFAIK currently this is the latest available firmware.

> Please run "ethtool -s eth0 | grep discard",

It should be "ethtool -S eth0 | grep discard" (grand S)

Best regards,

                         Krzysztof Olêdzki Received on 2008/12/19 08:29