I do not think you can see this with mode tcp, because of the lack of X-Forwarded-For. Your request could have been possible with mode http with the following settings:
in haproxy -> option forwardfor (I see you already have this enabled)
in squid -> (in this order)
logformat squid [%{%d/%m/%Y-%H:%M:%S}tl] %{X-Forwarded-For}>h through %>a %Ss/%03Hs %rm/%Hs %ru %Sh/%<A %mt Referer:%{Referer}>h
cache_access_log /var/log/squid3/access.log squid
and disable cache_access_log if it is enabled.
From the logformat above, the essential parts are: %{X-Forwarded-For}>h = read the X-Forwarded-For header in the http request %>a = display originating server or peer IP address
for more reference consult http://devel.squid-cache.org/customlog/logformat.html Do try this with mode tcp as I am currently unable to.
-----Original Message-----
From: David Young
Sent: Friday 11, March 03, 2011 2:05
To: haproxy#formilux.org
Subject: Re: How to deal with proxying shoutcast / ICY protocol
Works as advertised, thank you :)
One thing is now missing from my setup, and that's the ability to identify WHICH source IPs are accessing WHICH sites.
Without haproxy, I'm able to do this with squid's access.log, and with "mode http", I'm able to see this in the haproxy logs.
When I enable "mode tcp" though, to work around this silly shoutcast issue, I get source IPs _only_ in the haproxy.log, and my squid access log reports every request has having a source IP of my haproxy host.
Any ideas?
Thanks,
D
On 10/03/11 10:22 PM, David Young wrote:
> Hi Willy, Dan,
>
> Thanks for pointing out that I was still in http mode - that was an
> oversight, and should solve my problem for now. Long term, I'd rather
> use http mode over tcp for flexibility, but unfortunately I can't get
> away from having to support Shoutcast (and any other dodgy protocol you
> can think of!) :(
>
> Dan - I guess we're stuck making backend decisions on destination IP
> only - any reason you're using Squid 3.0 over 3.1?
>
> I'll test in the lab tomorrow and report back results :)
>
> D
>
>
>
> On 10/03/11 9:43 PM, Florescu, Dan Alexandru wrote:
>> Hi David,
>>
>> I'm kind of amazed that we have both the same problem ( http://www.formilux.org/archives/haproxy/1103/4397.html ) in almost the same circumstances, and we both asked for help at the same time :)
>>
>> I too am performing LB between proxies.
>> The issue with ICY is also in Squid. You need the upgrade_http0.9 ( http://www.squid-cache.org/Doc/config/upgrade_http0.9/ ) option. But this one works only with Squid 2.7 and 3.1, not with 3.0. I myself have a 3.0 for the main Squid, so I thought I would balance it with Haproxy (elegant solution) to a Squid 3.1.
>>
>> If I try streaming directly on the 3.1 Squid, it will work, but through Haproxy it will fail.
>> It will work through Haproxy if you enable mode tcp. Now, I see you have enabled mode tcp in frontend, but you didn't do this for the backend aswell. You also have a mode http in the defaults section, so if you don't explicitly specify it in the backend, it will get the default-specified mode (http in your case).
>> so: mode tcp in backend and you're ok.
>>
>> I myself have another issue, I cannot send the requests for ICY/radio streaming through my second squid because I don't have any relevant options to use in ACLs, so all requests are sent to the default_backend.
>>
>>
>> -----Original Message-----
>> From: David Young [
>> Sent: Thursday 10, March 03, 2011 4:36
>> To: haproxy#formilux.org
>> Subject: How to deal with proxying shoutcast / ICY protocol
>>
>> Hi folks,
>>
>> First-time poster here - we've been working on implementing haproxy to
>> perform load balancing between our backend squid proxies.
>>
>> I stumbled across an issue today when I realized that I can't listen to
>> streaming radio via my haproxy instance, but that it works fine if I
>> direct my request to one of by backend proxies directly.
>>
>> A bit of searching lead me to the issue of HTTP/1.1 vs ICY as the
>> probable cause
>> (http://www.squid-cache.org/Versions/v3/3.1/RELEASENOTES.html#ss2.10).
>>
>> I thought that if I configured haproxy in "mode tcp" instead of "mode
>> http", that it'd behave more like a simple loadbalancer, and just pass
>> the request onto my squid backend verbatim, but that doesn't seem to
>> have worked either.
>>
>> An example URL I'm having trouble with is http://66.225.205.47/;stream.mp3
>>
>> Even in "mode tcp", if I request that URL through by browser from
>> haproxy, I get a 502 error, whereas if I requested it directly from the
>> backend squid instance which services my request anyway, I get the
>> expected headers and stream.
>>
>> I thought I'd try telnetting to the haproxy / squid ports respectively,
>> to test, and I received differing output (below).
>>
>> Can anybody enlighten me?
>>
>> Many thanks,
>> David
>>
>> -- Relevant haproxy.cfg --
>>
>> #---------------------------------------------------------------------
>> # Global settings
>> #---------------------------------------------------------------------
>> global
>> log 127.0.0.1 local2
>> log /dev/log local2
>> chroot /var/lib/haproxy
>> pidfile /var/run/haproxy.pid
>> maxconn 4000
>> user haproxy
>> group haproxy
>> daemon
>>
>> #---------------------------------------------------------------------
>> # common defaults that all the 'listen' and 'backend' sections will
>> # use if not designated in their block
>> #---------------------------------------------------------------------
>> defaults
>> mode http
>> log global
>> option dontlognull
>> option http-server-close
>> option httplog
>>
>> # Log on start of connection, not completion
>> option logasap
>>
>> # log errors separately
>> option log-separate-errors
>>
>> option forwardfor
>> option redispatch
>> timeout connect 10000 # default 10 second time out if a backend is
>> not found
>> timeout client 300000
>> timeout server 300000
>> maxconn 60000
>> retries 3
>>
>>
>> #---------------------------------------------------------------------
>> # main frontend which proxys to the backends
>> #---------------------------------------------------------------------
>> frontend test_frontend *:5001
>> mode tcp
>> option tcpka
>> default_backend test_backend
>>
>>
>> backend test_backend
>> balance roundrobin
>> server localhost localhost:3128
>>
>>
>>
>> -- Telnet to haproxy infront of single squid tcp backend --
>>
>> DavidBook:~ davidy$ telnet webscan-dev.blahblah.net 5001
>> Trying x.x.x.x...
>> Connected to webscan-dev.blahblah.net.
>> Escape character is '^]'.
>> GET http://66.225.205.47/;stream.mp3
>> HTTP/1.0 502 Bad Gateway
>> Cache-Control: no-cache
>> Connection: close
>> Content-Type: text/html
>>
>> <html><body><h1>502 Bad Gateway</h1>
>> The server returned an invalid or incomplete response.
>> </body></html>
>> Connection closed by foreign host.
>> DavidBook:~ davidy$
>>
>>
>> -- Telnet to squid directly --
>>
>> DavidBook:~ davidy$ telnet webscan-dev.blahblah.net 3128
>> Trying x.x.x.x...
>> Connected to webscan-dev.blahblah.net.
>> Escape character is '^]'.
>> GET http://66.225.205.47/;stream.mp3
>> ICY 200 OK
>> Date: Thu, 10 Mar 2011 02:25:55 GMT
>> icy-notice1: <BR>This stream requires <a
>> href="http://www.winamp.com/">Winamp</a><BR>
>> icy-notice2: SHOUTcast Distributed Network Audio Server/Linux v1.9.8<BR>
>> icy-name: Family Friendly WBGL
>> icy-genre: Contemporary Christian
>> icy-url: http://www.wbgl.org
>> Content-Type: audio/mpeg
>> icy-pub: 1
>> icy-br: 96
>> X-Cache: MISS from webscan-dev.blahblah.net
>> X-Cache-Lookup: MISS from webscan-dev.blahblah.net:3128
>> Via: 0.0 webscan-dev.blahblah.net (squid/3.1.11)
>> Connection: close
>>
>> 00/Ȯt?????sƸD?,?4A?Q`F??BA????B??@(???u??kTC??????? <binary stream
>> continues...>
The information contained herein is intended for its addressee(s) only and it is privileged or otherwise confidential. Any unauthorized distribution, amendment or disclosure hereof is strictly forbidden by the law. Please find complete and translated versions at http://www.rompetrol.com/disclaimer.html Received on 2011/03/11 09:24
This archive was generated by hypermail 2.2.0 : 2011/03/11 09:30 CET