Re: Connection limiting & Sorry servers

Hi Bostjan,

On Mon, Aug 03, 2009 at 08:51:09AM +0200, Bo??tjan Mer??un wrote:
> > I really don't know why you're limiting on the number of requests
> > per second. It is not the proper way to do this at all. In fact,
> > you should *only* need to play with the server's maxconn parameter,
> > as there should be no reason your servers would be sensible to a
> > number of requests per second.
>
> I don't know why limiting number of requests per second wouldn't be
> proper way to limit the traffic?

yes it precisely is a way of limiting the traffic.

> During high load, we can get few thousand requests per second. Servers
> can't handle that.

why are you saying that ? Except for rare cases of huge bugs, a server is not limited in requests per second. At full speed, it will simply use 100% of the CPU, which is why you bought it after all. When a server dies, it's almost always because a limited resource has been exhausted, and most often this resource is memory. In some cases, it may be other limits such as sockets, file descriptors, etc... which cause some unexpected exceptions not to be properly caught.

> If I only set max connections for every server, they
> won't even get there. With for example 50 new connections at the same
> time per server, I think they will die before reaching the limit.

Then set the limit lower. If you're sure they can't handle 50 concurrent connections, it's exactly the reason you must set a maxconn below this value to protect them.

> The thing is we usually have events that start at a certain time and
> users know that. The event is also disabled until that time so sometimes
> we really get huge amount of connections at the beginning.

I'm well aware of the problem, many sites have the same. The queuing mechanism in haproxy was developped exactly for that. The first user was a gaming site which went from 50 req/s to 10000 req/s on patch days. They too thought their servers could not handle that, while it was just a matter of concurrent connections once again. By enabling the queueing mechanism, they could sustain the 10000 req/s with only a few hundred concurrent connections.

> If there is some mechanism inside HAProxy, that would help us survive
> such cases, please let me know.
>
> Besides, the contractor thinks that getting waiting room is better user
> experience than waiting for site to open.

with the queue, you're not particularly waiting for the site to open, you access it normally, it's just that you don't rush on it all at once.

> That is why I wished to let certain amount of people to site and show
> everybody else waiting room.

The problem with the waiting room is that only the application knows how many people are on the site. You can't guess that from TCP connection counts. And unfortunately, you will randomly accept then reject users inside a same session. And by doing that, sometimes you'll send some users in the waiting room while there are free slots on the servers.

However, with the queue you have the ability to decide to send people to a waiting room when the queue is full and if the user has no application cookie for instance. That way you know it's a new user and you prefer to let him wait for the best moment.

> > > I would like to limit number of current users on real servers and amount
> > > of new connections that can be created per some time unit.
> >
> > This is the point I cannot agree with. I think you need to limit the
> > amount of concurrent connections on your servers. Otherwise, your
> > application is deadly broken, but that's not how you introduced it
> > first :-)
>
> If deadly broken means it does not take care of connection limiting,
> then it is deadly broken.

No, being dependant on connection limiting is normal. Being dependant on connections per second is abnormal.

> Application does not do any limiting, servers do when they stop
> responding :)

Once again I'm well aware of this problem :-)

> If our servers handle can 200 concurrent connections and I limit all
> sites so that the total doesn't exceed that, I have to limit each site
> to about 5 concurrent connections (about 40 sites at the moment). That
> means that instead of using as much resources as possible, the site with
> the most traffic would be using 5 connections only.

No, if your servers are shared between multiple sites, you can very well use a same backend for all those sites, or even for some groups of sites. Also, limiting on the number of requests per second will never prevent a slow site from saturating your 200 concurrent connections.

> Am I right?
> I am thinking about fullconn option now and since we make sure that
> there is only one site having high traffic at the moment, I might be
> able to use that?

The fullconn is only used in conjunction with the minconn, it only tells the algorithm what threshold must be considered full load.

Hoping this helps,
Willy Received on 2009/08/03 09:21