Re: Sending requests to servers that are marked as down?

Hi Kai,

On Wed, Dec 03, 2008 at 11:44:10PM +0000, Kai Krueger wrote:
> >Yes, it does. I remember why this one was not checked, it was because
> >I did not want to leave unserved requests in the queue. But now that
> >set_server_up() checks the queue, there should be no problem. Feel free
> >to add a test above it.
> >
> Ok, I can confirm that with your original patch and the additional check
> things work correctly now and requests are no longer sent to the
> disabled servers.

Nice! My problem now is to solve the conditions for this last missing "if". If we do it unconditionally, we have the problem you reported. If we do it conditionally, the server might leave some pending connections unprocessed in its queue if its minconn does not allow it to take at least one connection due to a very low load on the backend, reason why the test was removed. Then I need to do two three things :

• check if it is possible to have too low a minconn which will prevent the last request from being processed at low backend loads ;
• see how I can have a different process_queue() which only dequeues a server's own connections and not global ones.
• create a queue of servers which have unprocessed pending connections when minconn is in use, and quickly check that queue when a new connection comes in. Ideally, "may_dequeue" would compute the minimum number of backend connections needed to release new requests from the queue, so that we use that as a sorting key to avoid checking all servers when a new connection comes in.

I will probably not implement the third option in the stable release as it is a bit risky. However, using this in session_free() would be a good trade-off :

    if (may_dequeue(srv))

       process_queue(srv);
    else if (srv->minconn)

       process_own_queue(srv);

Very few people use minconn, and the consequence of draining requests on a down server is not _that_ dramatic. In fact, this fix will at least ensure that the server is not taking connections out of the global queue which could have been processed by other servers. It will still be an improvement upon current situations for these users.

> Together with our patch to mark servers unavailable each time they
> return a 503 to the client, things are finally running smoothly and
> haproxy is doing a good job at balancing between the available servers. :-)

That's excellent. I want to thank you a lot for your work at analysing the code and experimenting with fixes. That was really helpful !

I'll issue a new release soon (probably next week-end).

Cheers!
Willy Received on 2008/12/04 06:41