Re: HAProxy and FreeBSD EADDRINUSE

From: Piotr Rybicki <meritus#innervision.pl>
Date: Mon, 08 Dec 2008 23:12:18 +0100

Piotr Rybicki pisze:
>> setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, (char *) &one, sizeof(one));
>> setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, (char *) &one, sizeof(one));
>>
>> If it works, we'll add this for FreeBSD.
>>
>
> I have just added this line and recompiled. I'll share results in a few
> days.
>

Sorry, no bonus here.

tail /var/log/messages:
Dec 8 22:58:27 SERVER haproxy[25582]: Connect() failed for server XXX/YYY: local address already in use.

I have a few source config parameters set in a backend section. Let's say I have a 3 backends with source 1.1.1.1(/24) and 5 backends with source 1.1.2.1(/24).
There is no rule in which backend(source) this issue is triggered.

I've updated pf config to enable adaptive state timeout. I've raised mbuf clusters to a high value.

I believe i have no out-ot-mbuf issue here.

Some commands:

# netstat -m
10719/81441/92160 mbufs in use (current/cache/total)

6737/72121/78858/102400 mbuf clusters in use (current/cache/total/max)
6737/72111 mbuf+clusters out of packet secondary zone in use (current/cache)
3919/8881/12800/25600 4k (page size) jumbo clusters in use

(current/cache/total/max)
0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 32097K/200126K/232223K bytes allocated to network (current/cache/total)

0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)

0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile 0 calls to protocol drain routines

# netstat -ltn | grep TIME_WAIT |wc -l

    8232

# sysctl net.inet.ip.portrange

net.inet.ip.portrange.randomtime: 45
net.inet.ip.portrange.randomcps: 10
net.inet.ip.portrange.randomized: 1
net.inet.ip.portrange.reservedlow: 0
net.inet.ip.portrange.reservedhigh: 1023
net.inet.ip.portrange.hilast: 65535
net.inet.ip.portrange.hifirst: 49152
net.inet.ip.portrange.last: 65535
net.inet.ip.portrange.first: 1024
net.inet.ip.portrange.lowlast: 600
net.inet.ip.portrange.lowfirst: 1023

Not sure if this gives some clue here.

How can I help to track down this issue?

Best regards
Piotr Rybicki Received on 2008/12/08 23:12