Re: Linux routing performace

Hi James,

I would agree with jw. If your internal network is all on the same subnet, you don't need the second gateway. Now if you are routing to different subnets on the internal network, you could simply put route statements pointing those routes to use the internal router instead of adding a second gateway on the haproxy server.

For instance:

route add -net 192.168.1.16 netmask 255.255.255.240 gw 10.0.0.1

Joe

On Tue, May 3, 2011 at 10:39 PM, Jon Watte <jwatte#imvu.com> wrote:

> Does the internal network need a gateway at all?
>
> We run a very similar set-up, HAProxy listening on a public network, and
> forwarding TCP connections to servers on an internal network. Because all
> the servers are on the same 10/8 subnet, no default gateway is needed.
>
> Sincerely,
>
> jw
>
>
> Jon Watte, IMVU.com
> We're looking for awesome people! http://www.imvu.com/jobs/
>
>
>
>
> On Tue, May 3, 2011 at 7:41 AM, James Bardin <jbardin#bu.edu> wrote:
>
>> Hello,
>>
>> This isn't necessarily an haproxy question, but I'm having trouble
>> finding a good resource, so I'm hoping some of the other experienced
>> people on this list may be able to help.
>>
>> Setup:
>> I have a load balancer configuration that needs to me multi-homed
>> across a private and public network. Both networks have strict reverse
>> path checking, so packets must be routed out their corresponding
>> interface, instead of a single default (each interface essentially has
>> it's own default gateway).
>>
>> The public net is eth0, so it gets the real default gateway. The
>> routing rules take any private-net packets, and send them out the
>> correct interface, to the private-net gateway.
>>
>> ####
>> ip route add default via 10.0.0.1 dev eth1 table 10
>> ip rule add from 10.0.0.0/8 table 10
>> ####
>>
>> Result:
>> What I've noticed is that any traffic handled by this one routing
>> decision drops the overall throughput to about 30% (it also seems adds
>> about .5ms to the rtt). Haproxy can handle about 1.5Gb/s of tcp
>> traffic on the public network, but only about 500Mb/s through the
>> private (there's an even greater skew when I remove haproxy, because
>> my link is close to 3Gb/s). Adding another cpu, and using interrupt
>> coalescing reduced the system cpu time, and brought down the
>> context-switches, but didn't increase performance at all.
>>
>> Any other tuning options I might try? I'm running the latest RHEL5
>> kernel at the moment (I haven't tried bringing up new machines with a
>> newer kernel yet)
>>
>>
>> Thanks,
>>
>> --
>> James Bardin <jbardin#bu.edu>
>> Systems Engineer
>> Boston University IS&T
>>
>>
>
Received on 2011/05/04 04:50