Re: IIS times out on large post requests

On Friday 11 July 2008 13:40:38 Rainer Sabelka wrote:
> Hi Willy!
>
> On Thursday 10 July 2008 19:57:09 Willy Tarreau wrote:
> > On Thu, Jul 10, 2008 at 11:32:30AM +0200, Rainer Sabelka wrote:
> > > But I'll install the patched version on the production system in the
> > > evening - and we will see tomorrow if it solves the problem.
> >
> > fine.
>
> I sill see some POST requests with 502 status in haproxy's logfile. So it
> looks like the problem is still there.
>
> > > Is there a
> > > possibility to log the actual size of the request body, so that I can
> > > compare it with the content length?
> >
> > Unfortunately not, but it should not be too much complicated to add it
> > (just that it would change the log format). If you're willing to try it,
> > simply modify the logging function (http_sess_log I believe, from memory)
> > to include the session's byte counters. You already have the byte counter
> > for the response, simply add the request next to it. Keep in mind that it
> > accounts for the whole request, headers included. But that will
> > definitely help in your case.
>
> Thanks for this hint.
> I added now s->logs.bytes_in to the logging function (I've put it into the
> {}- section with the captured request headers in order to not mess up the
> logfile format too much).
> So this is what I get now:
>
> Jul 11 12:37:45 127.0.0.1 haproxy[15072]: 127.0.0.1:35623
> [11/Jul/2008:12:35:26.355] myapp_http myapp_default/web8 1282/0/1/-1/138917
> 502 204 - - SHVN 6/3/3/3 0/0 {212.95.182.2|67113|66749} "POST
> /AttachDocument.Asp?SID=726220&action=143&state=0&binaryattach=1&TaskID=NEW
>_1&TSK_ORD_OrderID=85735 HTTP/1.0"
>
> So, now it gets interesting:
> The content-length is 67113 but the request size is only 66749 bytes.
> This means that the body of the POST request is missing some bytes and IIS
> is pretty correct to wait for more data (and eventually time out).
>
> So, what does this mean? Did haproxy receive the complete request but
> didn't send all of it to the backend server? Or did haproxy receive an
> incomplete request?

I have just tried to telnet to my haproxy server and send a manually crafted POST request - but I sent only a part of the body and then let it time out. -> I got the same error messages in IIS's and haproxy's logfiles.

So my conclusion is the cause of my problem could be one of the following:

• haproxy drops some part of the request body
• apache (mod_proxy/mod_ssl) forwards an inclomplete request body
• the connection to the client stalled while transmitting the request body
• bug in IE-6 which causes it to send invalid post requests

(Note: now I know why I saw these 502 errors only on the IIS 6 backend servers but not on IIS 5:
If IIS 6 times out while receiving the body it just closes the connection. IIS 5 is somewhat different: 1. After sending the headers it answers with a "100 Continue" header. 2. When getting the body times out it answers with a "500 Server Error" page.)

Best regards,
-Rainer Received on 2008/07/11 15:55