Re: many TIME_WAIT connection with strange configuration

Hi Kevin,

On Wed, Mar 05, 2008 at 05:48:43PM +0100, Kevin Maziere - Amen wrote:
> Hi,
>
> For test purpose I'm testing a haproxy loop configuration, all on the
> same machine.
>
> 5 differents instance of webservers, on the same machine, listening on
> different IP on port 8080.
> And 5 differents haproxy listener on port 80
>
> What I want to do :
> having haproxy able to switch from an server to another backup server if
> one failed, and so on until the last instance :

looks like a funny idea :-)

> listen haproxy_1 192.168.0.1:80
> maxconn 1000
> mode http
> cookie JSESSIONID prefix
> balance roundrobin
> server s1 192.168.0.1:8080 cookie f1 check inter 2000 fall 4 rise 1
> server s2 192.168.0.2:8080 cookie f2 check inter 10000 fall 4 rise 1 backup
> server f3 192.168.0.3:8080 cookie f3 check inter 10000 fall 4 rise 1 backup
> server f4 192.168.100.4:8080 cookie f4 check inter 10000 fall 4 rise 1
> backup
> server f5 192.168.100.5:8080 cookie f5 check inter 10000 fall 4 rise 1
> backup
> option httpchk /test.php
> option forwardfor

Be very careful: you're using cookie prefix mode, and you have not set "option httpclose", so if both your client and server does keep-alive, you will have wrong cookies for all requests sent after the first one in a same connection.

> And the code he repeated like this for all instances.
>
> In fact I wanted to test if I configure my haproxy proxy server a a
> backup for other haproxy server, themselves as a backup for other haproxy
> instance ...will it work or do something really nasty.
>
> My haproxy server will loop on haproxy backup servers, which are looping
> on haproxy backup server... and so on (I hope I'm clear)

I think (or hope) I understand.

> In my case it works, If 4 of the five haproxy failed, the last one still
> works fine, and my test domain still answer, if all failed, haproxy
> return code 503 and that is fine.
>
> But what I Remarque is even with no traffic (no connection to any of the
> haproxy) the number of connection grow up... grow
> I saw such connections ( number depends on the time since haproxy
> was launched, and of course of the timeout of tcp session, but more than
> 1000 ...), with the appropriate netstat command :
>
> -------------------------------------------------------------------------
>
> tcp 0 0 192.168.100.1:8080 192.168.100.1:43304 TIME_WAIT
> 0 0 -
> tcp 0 0 192.168.100.1:43304 192.168.100.1:8080 IME_WAIT
> 0 0 -
>
> -------------------------------------------------------------------------
> and the same for other IP .2,.3,.4
> If I put a interval check lower, for example 2000 on my backup server,
> then the number of time wait connections grow up faster
>
> First I thought I was the check launched by haproxy, but if it was such
> check, I should see something like
> IP.1 IP.2
> IP.2 IP.1
>
> Could someone help me to understand what ? I know that using haproxy
> like the way I'm testing it is not really appropriate ... maybe it is the
> reason why (all such bad things on the same machine).

those ones are not connections, but terminated connections. The TIME_WAIT state is mandatory once a connection is finished, in order to catch late packets. It's most likely set to 60 seconds on your system, although it's still common to see 240s. You don't have to worry about them, they're very cheap (eg: on linux, they consume between 56 and 84 bytes depending on whether IPv6 is compiled in your kernel or not). I've already reached several millions of them without even a slowdown. Moreover, when a process needs to reuse the source port for the same destination, they are immediately purged and reassigned, so they don't block anything.

regards,
Willy Received on 2008/03/05 23:35