Re: SSL Load balancing question

i did some further debugging and also enabled the rsyslog on my ubuntu server but i am unable to find the root cause of this issue???

here is what my admin stat looks like (if you see the swebcluster/load1-app has all values 0)

webcluster   
    Queue    Session rate    Sessions    Bytes    Denied    Errors    Warnings    Server
Cur    Max    Limit    Cur    Max    Limit    Cur    Max    Limit    Total    LbTot    In    Out    Req    Resp    Req    Conn    Resp    Retr    Redis    Status    LastChk    Wght    Act    Bck    Chk    Dwn    Dwntme    Thrtle Frontend        1    32    -    1    6    2000    203        85538    1850676    0    0    0                    OPEN    load1-app    0    0    -    0    2        0    2    -    109    109    30470    1198316        0        0    0    0    0    2h30m UP    L7OK/200 in 1ms    1    Y    -    0    0    0s    - load2-app    0    0    -    0    32        0    6    -    72    12    37869    395381        0        0    0    0    0    2h30m UP    L7OK/200 in 73ms    1    Y    -    1    0    0s    - Backend    0    0        0    32        0    6    2000    181    121    85538    1850676    0    0        0    0    0    0    2h30m UP         2    2    0         0    0s   

swebcluster   
    Queue    Session rate    Sessions    Bytes    Denied    Errors    Warnings    Server
Cur    Max    Limit    Cur    Max    Limit    Cur    Max    Limit    Total    LbTot    In    Out    Req    Resp    Req    Conn    Resp    Retr    Redis    Status    LastChk    Wght    Act    Bck    Chk    Dwn    Dwntme    Thrtle Frontend        0    5    -    0    6    2000    28        61312    267759    0    0    0                    OPEN    load1-app    0    0    -    0    0        0    0    -    0    0    0    0        0        0    0    0    0    2h30m UP    L7OK/200 in 1ms    2    Y    -    0    0    0s    - load2-app    0    0    -    0    5        0    6    -    28    28    61312    267759        0        0    0    0    0    2h30m UP    L7OK/200 in 75ms    2    Y    -    0    0    0s    - Backend    0    0        0    5        0    6    2000    28    28    61312    267759    0    0        0    0    0    0    2h30m UP         4    2    0         0    0s   

the output from my /var/log/haproxy.log is

Mar  2 20:03:04 localhost haproxy[21654]: 105:59775 [02/Mar/2011:20:03:04.402] webcluster webcluster/load2-app 1/15/82 5214 -- 6/5/5/5/0 0/0
Mar  2 20:03:04 localhost haproxy[21654]: 105:59774 [02/Mar/2011:20:03:04.401] webcluster webcluster/load2-app 0/17/85 808 -- 5/4/4/4/0 0/0
Mar  2 20:03:04 localhost haproxy[21654]: 105:59773 [02/Mar/2011:20:03:04.401] webcluster webcluster/load2-app 0/17/87 804 -- 4/3/3/3/0 0/0
Mar  2 20:03:04 localhost haproxy[21654]: 105:59776 [02/Mar/2011:20:03:04.403] webcluster webcluster/load2-app 3/12/85 1432 -- 3/2/2/2/0 0/0
Mar  2 20:03:04 localhost haproxy[21654]: 105:59771 [02/Mar/2011:20:03:04.397] webcluster webcluster/load2-app 1/21/92 4874 -- 2/1/1/1/0 0/0
Mar  2 20:03:04 localhost haproxy[21654]: 105:59772 [02/Mar/2011:20:03:04.397] webcluster webcluster/load2-app 4/17/204 20075 -- 1/0/0/0/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59778 [02/Mar/2011:20:03:05.132] webcluster webcluster/load2-app 3/2/68 556 -- 6/5/5/5/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59780 [02/Mar/2011:20:03:05.138] webcluster webcluster/load2-app 4/36/104 556 -- 6/5/5/5/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59779 [02/Mar/2011:20:03:05.135] webcluster webcluster/load2-app 2/41/107 1325 -- 5/4/4/4/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59781 [02/Mar/2011:20:03:05.138] webcluster webcluster/load2-app 9/31/107 1188 -- 4/3/3/3/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59782 [02/Mar/2011:20:03:05.142] webcluster webcluster/load2-app 3/33/103 4392 -- 3/2/2/2/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59777 [02/Mar/2011:20:03:05.132] webcluster webcluster/load2-app 0/5/145 13095 -- 2/1/1/1/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59784 [02/Mar/2011:20:03:05.225] webcluster webcluster/load2-app 1/27/92 482 -- 6/5/5/5/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59788 [02/Mar/2011:20:03:05.287] webcluster webcluster/load2-app 2/9/76 529 -- 6/5/5/5/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59787 [02/Mar/2011:20:03:05.286] webcluster webcluster/load2-app 3/9/77 481 -- 5/4/4/4/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59786 [02/Mar/2011:20:03:05.285] webcluster webcluster/load2-app 3/10/79 481 -- 4/3/3/3/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59785 [02/Mar/2011:20:03:05.278] webcluster webcluster/load2-app 6/13/85 482 -- 3/2/2/2/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59792 [02/Mar/2011:20:03:05.312] webcluster webcluster/load2-app 1/25/91 532 -- 3/2/1/1/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59793 [02/Mar/2011:20:03:05.343] webcluster webcluster/load2-app 0/35/91 323 -- 6/5/5/5/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59797 [02/Mar/2011:20:03:05.407] webcluster webcluster/load2-app 0/8/76 391 -- 6/5/4/4/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59794 [02/Mar/2011:20:03:05.404] webcluster webcluster/load2-app 0/11/79 323 -- 5/4/3/3/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59795 [02/Mar/2011:20:03:05.404] webcluster webcluster/load2-app 2/9/80 1115 -- 4/3/3/3/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59798 [02/Mar/2011:20:03:05.428] webcluster webcluster/load2-app 1/29/127 9343 -- 6/5/5/5/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59796 [02/Mar/2011:20:03:05.407] webcluster webcluster/load2-app 0/8/176 30906 -- 5/4/4/4/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59802 [02/Mar/2011:20:03:05.515] webcluster webcluster/load2-app 0/22/90 1110 -- 5/4/4/4/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59800 [02/Mar/2011:20:03:05.512] webcluster webcluster/load2-app 0/25/139 10066 -- 4/3/3/3/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59799 [02/Mar/2011:20:03:05.481] webcluster webcluster/load2-app 2/12/181 26587 -- 3/2/2/2/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59803 [02/Mar/2011:20:03:05.585] webcluster webcluster/load2-app 3/31/104 532 -- 5/4/4/4/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59801 [02/Mar/2011:20:03:05.512] webcluster webcluster/load2-app 3/22/239 50544 -- 6/5/5/5/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59805 [02/Mar/2011:20:03:05.681] webcluster webcluster/load2-app 0/20/78 533 -- 5/4/4/4/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59806 [02/Mar/2011:20:03:05.684] webcluster webcluster/load2-app 1/16/75 514 -- 4/3/3/3/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59804 [02/Mar/2011:20:03:05.669] webcluster webcluster/load2-app 1/30/90 384 -- 3/2/2/2/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59808 [02/Mar/2011:20:03:05.712] webcluster webcluster/load2-app 0/26/142 447 -- 3/2/1/1/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59807 [02/Mar/2011:20:03:05.711] webcluster webcluster/load2-app 0/27/143 514 -- 2/1/0/0/0 0/0
Mar  2 20:03:05 localhost haproxy[21654]: 105:59809 [02/Mar/2011:20:03:05.781] webcluster webcluster/load2-app 74/3/140 448 -- 1/0/0/0/0 0/0
Mar  2 20:03:06 localhost haproxy[21654]: 105:59812 [02/Mar/2011:20:03:05.940] webcluster webcluster/load2-app 0/38/102 477 -- 3/2/2/2/0 0/0
Mar  2 20:03:06 localhost haproxy[21654]: 105:59811 [02/Mar/2011:20:03:05.938] webcluster webcluster/load2-app 2/38/106 449 -- 2/1/1/1/0 0/0
Mar  2 20:03:06 localhost haproxy[21654]: 105:59810 [02/Mar/2011:20:03:05.936] webcluster webcluster/load2-app 1/41/144 6066 -- 1/0/0/0/0 0/0
Mar  2 20:03:06 localhost haproxy[21654]: 105:59813 [02/Mar/2011:20:03:06.109] webcluster webcluster/load2-app 2/25/89 1056 -- 1/0/0/0/0 0/0
Mar  2 20:03:19 localhost haproxy[21654]: 11:41581 [02/Mar/2011:20:02:59.467] swebcluster swebcluster/load2-app 0/32/19713 4366 -- 0/0/0/0/0 0/0
Mar  2 20:04:15 localhost haproxy[21654]: 124:50153 [02/Mar/2011:20:04:15.298] webcluster webcluster/load1-app 35/0/200 30906 -- 1/0/0/0/0 0/0
Mar  2 20:04:15 localhost haproxy[21654]: 124:50154 [02/Mar/2011:20:04:15.777] webcluster webcluster/load1-app 40/1/156 26587 -- 1/0/0/0/0 0/0
Mar  2 20:04:17 localhost haproxy[21654]: 124:50157 [02/Mar/2011:20:04:17.219] webcluster webcluster/load1-app 74/1/235 50544 -- 1/0/0/0/0 0/0
Mar  2 20:04:17 localhost haproxy[21654]: 105:49229 [02/Mar/2011:20:04:17.877] webcluster webcluster/load2-app 0/20/84 808 -- 6/5/5/5/0 0/0
Mar  2 20:04:17 localhost haproxy[21654]: 105:49231 [02/Mar/2011:20:04:17.878] webcluster webcluster/load2-app 2/18/90 804 -- 5/4/4/4/0 0/0
Mar  2 20:04:17 localhost haproxy[21654]: 105:49227 [02/Mar/2011:20:04:17.870] webcluster webcluster/load2-app 4/23/107 5214 -- 4/3/3/3/0 0/0
Mar  2 20:04:17 localhost haproxy[21654]: 105:49232 [02/Mar/2011:20:04:17.878] webcluster webcluster/load2-app 2/18/104 1432 -- 3/2/2/2/0 0/0
Mar  2 20:04:17 localhost haproxy[21654]: 105:49230 [02/Mar/2011:20:04:17.878] webcluster webcluster/load2-app 1/19/106 4874 -- 2/1/1/1/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49228 [02/Mar/2011:20:04:17.876] webcluster webcluster/load2-app 1/21/168 20075 -- 1/0/0/0/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49236 [02/Mar/2011:20:04:18.217] webcluster webcluster/load2-app 1/40/111 556 -- 6/5/5/5/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49241 [02/Mar/2011:20:04:18.234] webcluster webcluster/load2-app 1/22/108 481 -- 5/4/4/4/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49239 [02/Mar/2011:20:04:18.234] webcluster webcluster/load2-app 0/24/109 1325 -- 4/3/3/3/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49240 [02/Mar/2011:20:04:18.234] webcluster webcluster/load2-app 1/23/111 482 -- 3/2/2/2/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49237 [02/Mar/2011:20:04:18.230] webcluster webcluster/load2-app 1/27/115 4392 -- 2/1/1/1/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49238 [02/Mar/2011:20:04:18.231] webcluster webcluster/load2-app 2/24/163 13095 -- 6/5/5/5/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49242 [02/Mar/2011:20:04:18.348] webcluster webcluster/load2-app 1/29/51 532 -- 5/4/4/4/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49243 [02/Mar/2011:20:04:18.363] webcluster webcluster/load2-app 3/12/82 323 -- 6/5/5/5/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49246 [02/Mar/2011:20:04:18.367] webcluster webcluster/load2-app 1/10/80 391 -- 5/4/4/4/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49244 [02/Mar/2011:20:04:18.367] webcluster webcluster/load2-app 1/10/83 1115 -- 4/3/3/3/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49251 [02/Mar/2011:20:04:18.493] webcluster webcluster/load2-app 3/2/51 1110 -- 6/5/5/5/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49247 [02/Mar/2011:20:04:18.428] webcluster webcluster/load2-app 0/30/126 9343 -- 5/4/4/4/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49245 [02/Mar/2011:20:04:18.367] webcluster webcluster/load2-app 1/10/198 30906 -- 4/3/3/3/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49249 [02/Mar/2011:20:04:18.466] webcluster webcluster/load2-app 2/29/103 10066 -- 4/3/2/2/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49248 [02/Mar/2011:20:04:18.429] webcluster webcluster/load2-app 1/28/200 26587 -- 5/4/3/3/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49250 [02/Mar/2011:20:04:18.470] webcluster webcluster/load2-app 5/22/208 50544 -- 5/4/3/3/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49254 [02/Mar/2011:20:04:18.598] webcluster webcluster/load2-app 2/18/92 514 -- 4/3/2/2/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49253 [02/Mar/2011:20:04:18.578] webcluster webcluster/load2-app 1/38/117 384 -- 3/2/1/1/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49256 [02/Mar/2011:20:04:18.650] webcluster webcluster/load2-app 3/5/65 448 -- 3/2/1/1/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49257 [02/Mar/2011:20:04:18.699] webcluster webcluster/load2-app 1/37/146 6066 -- 2/1/0/0/0 0/0
Mar  2 20:04:18 localhost haproxy[21654]: 105:49252 [02/Mar/2011:20:04:18.569] webcluster webcluster/load2-app 306/22/389 533 -- 1/0/0/0/0 0/0
Mar  2 20:04:21 localhost haproxy[21654]: 105:49255 [02/Mar/2011:20:04:21.590] webcluster webcluster/load2-app 1/21/89 477 -- 1/0/0/0/0 0/0
Mar  2 20:04:21 localhost haproxy[21654]: 105:49258 [02/Mar/2011:20:04:21.713] webcluster webcluster/load2-app 14/11/88 1056 -- 1/0/0/0/0 0/0
Mar  2 20:04:32 localhost haproxy[21654]: 11:41766 [02/Mar/2011:20:04:11.209] swebcluster swebcluster/load2-app 0/4/21579 4366 -- 0/0/0/0/0 0/0
Mar  2 20:04:59 localhost haproxy[21654]: 140:45978 [02/Mar/2011:20:04:59.439] webcluster webcluster/load1-app 0/0/137 210 -- 0/0/0/0/0 0/0
Mar  2 20:04:59 localhost haproxy[21654]: 140:45980 [02/Mar/2011:20:04:59.758] webcluster webcluster/load1-app 41/0/66 26587 -- 0/0/0/0/0 0/0
Mar  2 20:05:40 localhost haproxy[21654]: 105:59828 [02/Mar/2011:20:05:40.241] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0
Mar  2 20:05:42 localhost haproxy[21654]: 105:59829 [02/Mar/2011:20:05:42.642] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0
Mar  2 20:05:44 localhost haproxy[21654]: 105:59830 [02/Mar/2011:20:05:44.969] webcluster webcluster/<NOSRV> -1/-1/1 12338 PR 0/0/0/0/0 0/0
Mar  2 20:05:47 localhost haproxy[21654]: 105:59831 [02/Mar/2011:20:05:47.551] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0
Mar  2 20:05:48 localhost haproxy[21654]: 105:59832 [02/Mar/2011:20:05:48.751] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0
Mar  2 20:05:55 localhost haproxy[21654]: 105:59833 [02/Mar/2011:20:05:55.856] webcluster webcluster/<NOSRV> -1/-1/1 12338 PR 0/0/0/0/0 0/0

• On Wed, 3/2/11, Amol <mandm_zoom#yahoo.com> wrote:

From: Amol <mandm_zoom#yahoo.com>
Subject: Re: SSL Load balancing question To: haproxy#formilux.org
Date: Wednesday, March 2, 2011, 3:52 PM

I have a question regarding this config of haproxy, i see that all my connections are only going to app1 and app2 has no connections at all, i changed it last night from balance roundrobin to balance source, do i have to flush any files, etc. Why would this be occuring?

listen swebcluster 1.1.1.8:8000
        mode tcp
        balance source
        option ssl-hello-chk
        option httpchk HEAD /check.txt HTTP/1.0
        server app1 159:80 maxconn 1024 check
        server app2 148:80 maxconn 1024 check

• On Sat, 2/26/11, Amol <mandm_zoom#yahoo.com> wrote:

From: Amol <mandm_zoom#yahoo.com>
Subject: SSL Load balancing question
To: haproxy#formilux.org
Date: Saturday, February 26, 2011, 4:06 PM

So i have a haproxy 1.4 setup with the following configuration

global
    log 127.0.0.1    local0
    log 127.0.0.1    local1 notice
    maxconn 20000
    ulimit-n 65536
    user haproxy
    group haproxy
    daemon
        stats socket /var/run/haproxy/haproxy.sock mode 0600 level admin

defaults
    log    global
    option    tcplog
    option    dontlognull
    retries    3
    option redispatch
    contimeout    5000
    clitimeout    50000
    srvtimeout    200000

listen swebcluster 1.1.42.18:800
        mode tcp
        balance roundrobin
        option ssl-hello-chk
        option httpchk HEAD /check.txt HTTP/1.0
        server p-server-app1 1.1.4.59:80  maxconn 5000 check
        server p-server-app2 1.1.7.148:80 maxconn 5000 check

one thing i have noticed is that when i run a long running request on the load balanced url the whole setup just hangs, like its waiting on this one request to complete, (yes my application does frequent request and since it is roundrobin may be it is going to the server that is already handling one request?

my question is if there is a better configuration to handle this, so my app can use the same server and other users are not affected by this 1 request? the cookie setup is not possible in tcp correct? Received on 2011/03/03 02:14