RE: Quick question on logging from HaProxy.

Personally I prefer virtual machines or zones or whatever to chroot jails. Granted, if an app is hacked they have more user-land utilities to do more evil, but it's also a lot easier to separate apps from another than chrooting non chroot friendly apps and still being able to put more than one service on a box. Anyways, I don't want to start a religious war of the virtues of chroot, just mentioning it's not important to everyone, but I understand it is important to some. Also you can have syslogd listen on a total of 20 sockets, so unless you want more than 19 chroot apps, the chroot issue is moot.

Is this an either/or situation? Is there a problem with supporting both methods and letting it be picked from the config file? Perhaps with a warning in the documentation that some versions of syslog will choke on the load and is not recommended for high traffic sites, and so they should use UDP for maximum performance.

> -----Original Message-----
> From: Willy Tarreau [mailto:w#1wt.eu]
> Sent: Wednesday, October 24, 2007 10:57 AM
> To: Lauro, John
> Cc: haproxy#formilux.org
> Subject: Re: Quick question on logging from HaProxy.
>
> On Wed, Oct 24, 2007 at 09:43:37AM -0400, Lauro, John wrote:
> > Perhaps the problem is using UDP instead of sockets. UDP is
> > unreliable compared to sockets
>
> it's not much a problem of reliability, it's a problem of
performance
> I've been experiencing. Having syslogd spin at 80% CPU at 1000
lines/s
> is not acceptable.
>
> > and is another reason it would be nice if haproxy supported it.
> > If you switch to sockets I am sure you will
> > find you can log much more traffic more reliably.
>
> There are two problems to this :
> - if the process is chrooted, it cannot access sockets outside its
> chroot, and syslogd cannot use anything about /dev/log.
Obviously
> noone would consider chrooting haproxy into /dev :-)
>
> - the congestion problem will simply be forwarded before the
socket,
> which is on haproxy's side. Your application which reliably logs
> to a connected socket either pauses when the socket is full, or
> supports a log buffer. While a log buffer is desirable, it
should
> not be an excuse for the server to read slowly.
>
> Another very simple syslog I have adapted from busybox is able to
> forward 10k logs/s from UDP to UDP with less than 20% CPU on a
single
> P4/3.2GHz. I find this a good starting point.
>
> > I have one host that sometimes peeks in at tens of thousands of
logs a
> > second via syslog over a socket and never misses a beat. However,
I
> > cheat a little more than just using -, and have it log to /dev/shm
> > (essentially a ram disk) and have a cron job that runs once a
minute
> > to rotate and then consolidate it...
>
> It may depend on the write pattern, I don't know.
>
> Regards,
> Willy
Received on 2007/10/24 17:35