Re: many TIME_WAIT connection with strange configuration

From: Kevin Maziere - Amen <kevin.maziere#amen.fr>
Date: Thu, 06 Mar 2008 09:59:58 +0100


Willy Tarreau a écrit :

> Hi Kevin,
> 
> On Wed, Mar 05, 2008 at 05:48:43PM +0100, Kevin Maziere - Amen wrote:

>> Hi,
>>
>> For test purpose I'm testing a haproxy loop configuration, all on the
>> same machine.
>>
>> 5 differents instance of webservers, on the same machine, listening on
>> different IP on port 8080.
>> And 5 differents haproxy listener on port 80
>>
>> What I want to do :
>> having haproxy able to switch from an server to another backup server if
>> one failed, and so on until the last instance :
>
> looks like a funny idea :-)

Yes I agree with you, but very usefull for my test purpose :)

> 

>> listen haproxy_1 192.168.0.1:80
>> maxconn 1000
>> mode http
>> cookie JSESSIONID prefix
>> balance roundrobin
>> server s1 192.168.0.1:8080 cookie f1 check inter 2000 fall 4 rise 1
>> server s2 192.168.0.2:8080 cookie f2 check inter 10000 fall 4 rise 1 backup
>> server f3 192.168.0.3:8080 cookie f3 check inter 10000 fall 4 rise 1 backup
>> server f4 192.168.100.4:8080 cookie f4 check inter 10000 fall 4 rise 1
>> backup
>> server f5 192.168.100.5:8080 cookie f5 check inter 10000 fall 4 rise 1
>> backup
>> option httpchk /test.php
>> option forwardfor
> 
> Be very careful: you're using cookie prefix mode, and you have not set
> "option httpclose", so if both your client and server does keep-alive,
> you will have wrong cookies for all requests sent after the first one
> in a same connection.
> 

this option is included in the default part of the conf, I forgot to write it here :)
>> And the code he repeated like this for all instances.
>>
>> In fact I wanted to test if I configure my haproxy proxy server a a
>> backup for other haproxy server, themselves as a backup for other haproxy
>> instance ...will it work or do something really nasty.
>>
>> My haproxy server will loop on haproxy backup servers, which are looping
>> on haproxy backup server... and so on (I hope I'm clear)

> 
> I think (or hope) I understand.
> 

>> In my case it works, If 4 of the five haproxy failed, the last one still
>> works fine, and my test domain still answer, if all failed, haproxy
>> return code 503 and that is fine.
>>
>> But what I Remarque is even with no traffic (no connection to any of the
>> haproxy) the number of connection grow up... grow
>> I saw such connections ( number depends on the time since haproxy
>> was launched, and of course of the timeout of tcp session, but more than
>> 1000 ...), with the appropriate netstat command :
>>
>> -------------------------------------------------------------------------
>>
>> tcp 0 0 192.168.100.1:8080 192.168.100.1:43304 TIME_WAIT
>> 0 0 -
>> tcp 0 0 192.168.100.1:43304 192.168.100.1:8080 IME_WAIT
>> 0 0 -
>>
>> -------------------------------------------------------------------------
>> and the same for other IP .2,.3,.4
>> If I put a interval check lower, for example 2000 on my backup server,
>> then the number of time wait connections grow up faster
>>
>> First I thought I was the check launched by haproxy, but if it was such
>> check, I should see something like
>> IP.1 IP.2
>> IP.2 IP.1
>>
>> Could someone help me to understand what ? I know that using haproxy
>> like the way I'm testing it is not really appropriate ... maybe it is the
>> reason why (all such bad things on the same machine).
> 
> those ones are not connections, but terminated connections. The TIME_WAIT
> state is mandatory once a connection is finished, in order to catch late
> packets. It's most likely set to 60 seconds on your system, although it's
> still common to see 240s. You don't have to worry about them, they're
> very cheap (eg: on linux, they consume between 56 and 84 bytes depending
> on whether IPv6 is compiled in your kernel or not). I've already reached
> several millions of them without even a slowdown. Moreover, when a process
> needs to reuse the source port for the same destination, they are immediately
> purged and reassigned, so they don't block anything.
> 

Well thanks for this very good explanation, i misunderstood this time of connection, that clear now

> regards,
> Willy
> 
> 

Thanks for your quick answser
Kevin Received on 2008/03/06 09:59

This archive was generated by hypermail 2.2.0 : 2008/03/06 10:00 CET