Re: HAproxy performance on virtual sys. KVM

On Sat, Jul 10, 2010 at 11:25:46AM +0200, Konrad wrote:
> I have nowhere iptables enabled. Also mentioned in many posts
> "conntrack" isn't enabled anywhere.

OK

> I tested it now like this (same example test with ab: ab -c 100 -n 1000
> http://IP_addr/check.txt):
> vm5 (additional vm from which I made tests) -> vm11 (here is one of
> Apache conf.)
> ab: ~ Requests per second: 2872.94 [#/sec]
> vm5 -> vm21 (here is one of Apache conf.)
> ab: ~ Requests per second: 3089.87 [#/sec]
> vm5 -> 192.168.0.111 (HA, connects to vm11 and vm21)
> ab: ~ Requests per second: 1677.04 [#/sec]
>
> the haproxy.conf (changed for test):
> server webA 192.168.0.52:80 weight 4 cookie A maxconn 255 check inter 3000
> server webB 192.168.0.62:80 weight 24 cookie B maxconn 255 check inter 3000
>
> The test "page" is empty file check.txt (tested with links shows "OK" on
> every configuration).
> I changed also the sysctl.conf (vm1):
> net.ipv4.tcp_synack_retries = 3
> net.ipv4.conf.default.rp_filter=1
> net.ipv4.conf.all.rp_filter=1
> net.core.rmem_max = 8738000
> net.core.wmem_max = 6553600
> net.ipv4.tcp_rmem = 8192 873800 8738000
> net.ipv4.tcp_wmem = 4096 655360 6553600
> net.ipv4.tcp_tw_reuse = 1
> net.ipv4.tcp_max_tw_buckets = 360000
> vm.min_free_kbytes = 65536
> vm.swappiness = 0
>
> I just can't find out why HA is in my test configuration a bit slow.

In fact, I find that your numbers are consistent : when you connect from ab to apache directly, you go out of the VM once and you see 3000 hits/s. When you connect from ab to apache via haproxy, you go out of the VM twice and you see half of the performance, which really makes me insist on the host OS configuration. Your performance halves every time a packet leaves one VM to enter another one. Such numbers are already extremely low, 3000 hits/s is what haproxy can do on a PCEngines ALIX running a fanless Geode processor at 500 MHz ! Even VMWare and Xen are a lot faster. Since I've always heard KVM was faster than them, I'm having some doubts about the overall config !

Are you running on emulated network drivers instead of native ones ? It could explain why packet processing would be that slow.

Regards,
Willy Received on 2010/07/10 12:48