Re: TProxy 4

Hello,

On Mon, Aug 04, 2008 at 10:39:37AM -0600, Caleb Anthony wrote:
> Hello all.
>
> For the past few days I have been trying to setup HAProxy 1.3.15.2 with
> TProxy 4.1 so that my backend servers will see the original clients IP
> address. This is mostly for logging purposes, and enabling X-Forwarded-For
> isn't an option in my environment because I need HTTP Keep-Alive enabled on
> my servers (turning it off with option httpclose breaks Integrated Windows
> Authentication on IIS).
>
> Here is what I have so far:
>
> I patched my kernel with the latest patches from BalaBit. I originally tried
> the tproxy-kernel-2.6.24-20080602-165651-1212418611.tar.bz2 patch with
> kernel 2.6.24.5, and I also tried the
> tproxy-kernel-2.6.25-20080519-165031-1211208631.tar.bz2 patch with kernel
> 2.6.25.

I really need to try those patches again :-)

> I re-compiled HAproxy with USE_LINUX_TPROXY=1
>
> Lastly I patched and recompiled iptables 1.4.0 with the
> tproxy-kernel-2.6.25-20080519-165031-1211208631.tar.bz2 patch. I'm not
> entirely sure if this is really needed, other than to load the proper kernel
> modules.
>
> Once all the patching was done, I tried loading the TProxy modules manually
> with modprobe, and I also tried setting some iptables rules that also loaded
> the kernel modules.
>
> I know the modules are loaded and working because /var/log/messages displays
> the following after the modules are loaded:
> kernel: NF_TPROXY: Transparent proxy support initialized, version 4.1.0
> kernel: NF_TPROXY: Copyright (c) 2006-2007 BalaBit IT Ltd.
>
> I also set the following sysctl values (not sure if these are needed
> either):
> echo 1 > /proc/sys/net/ipv4/ip_forward
> echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind
>
> Also I added the following line to my haproxy.cfg:
> source 0.0.0.0 usesrc clientip

With tproxy v2, it was not possible to use source 0.0.0.0. Could you try to put your LB's address here ? You need to set up the address from which you can reach the backend server. Tproxy needed this in order to create its internal NAT tables.

> When a client connects, I can see the connection in the HAProxy debug
> output, but the browser just hangs for a little bit, then eventually HAProxy
> returns a HTTP 503 No server is available to handle this request.
>
> Here is the debug output from HAProxy:
> Available polling systems :
> sepoll : pref=400, test result OK
> epoll : pref=300, test result OK
> poll : pref=200, test result OK
> select : pref=150, test result OK
> Total: 4 (4 usable), will use sepoll.
> Using sepoll() as the polling mechanism.
> 00000000:Frontend.accept(0004)=0005 from [x.x.x.x:4851]
> 00000000:Frontend.clireq[0005:ffff]: GET /test/ HTTP/1.1
> 00000000:Frontend.clihdr[0005:ffff]: Accept: */*
> 00000000:Frontend.clihdr[0005:ffff]: Accept-Language: en-us
> 00000000:Frontend.clihdr[0005:ffff]: Accept-Encoding: gzip, deflate
> 00000000:Frontend.clihdr[0005:ffff]: User-Agent: Mozilla/4.0 (compatible;
> MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322;
> InfoPath.1; .NET CLR 2.0.50727)
> 00000000:Frontend.clihdr[0005:ffff]: Host: xxxxx
> 00000000:Frontend.clihdr[0005:ffff]: Connection: Keep-Alive
> 00000000:Frontend.clihdr[0005:ffff]: Cache-Control: no-cache
> 00000000:Default.srvcls[0005:0006]
> 00000000:Default.srvcls[0005:0006]
> 00000000:Default.srvcls[0005:0006]
> 00000000:Default.clicls[0005:0006]
> 00000000:Default.srvcls[0005:0006]
> 00000000:Default.closed[0005:0006]
>
> Removing "source 0.0.0.0 usesrc clientip" from my config makes HAProxy work
> fine (except with no transparent proxy functionality of course).

It would be nice if you could bring an strace output of the process during such a test, as well as a tcpdump trace of the interface with the server. That will tell us if the outgoing packets are correct (if they are there at all), and if the response traffic is correctly interpreted (if any at all).

> Here is my haproxy.cfg:

at first glance, it looks fine.

> global
> debug
>
> defaults
> balance source
> mode http
> timeout client 60s
> timeout connect 5s
> timeout http-request 10s
> timeout server 60s
>
> frontend Frontend
> bind 0.0.0.0:80
> default_backend Default
>
> backend Default
> server Web_Server_1 xxxxx:80 source 0.0.0.0 usesrc client
> Any help or direction would be appreciated.
>
> Thanks.

Regards,
Willy Received on 2008/08/05 07:37