Re: Reducing I/O load of logging

Hi John,

Can you explain what you mean by making sure there's a dash in the name? Is this for something like syslog via /var/log/messages?

We've had some major issues when logrotate kicks in, knocking haproxy offline (it reports most of the back-end servers as down and starts throwing 503s). It's a busy site with 80 servers configured in the backend. Reducing logrotate to 3 hour intervals shortened the downtimes but it still happened. It does the same thing if I try to grep the log.

I've had to turn off logrotate for now...

On Wed, Feb 11, 2009 at 10:01 AM, John Lauro <john.lauro#covenanteyes.com> wrote:
> It shouldn't be too hard for a machine to keep up with logging. How are you
> logging? standard syslog? Make sure you have - in front of the filename?
> How connections per second are you logging?
>
>
>
> Haven't done it with Haproxy, but have with other things that generate tons
> of logs…
>
>
>
> what you could do is dump the logs (don't forget the – as part of the file
> name) to /dev/shm/ (assuming linux), and then rotate the logs once a minute
> and process them… That way, you will not have any disk I/O from the logs,
> but would increase memory requirements.
>
>
>
>
>
>
>
> From: James Brady [mailto:james.colin.brady#gmail.com]
> Sent: Wednesday, February 11, 2009 12:14 PM
> To: haproxy#formilux.org
> Subject: Reducing I/O load of logging
>
>
>
> The machine we run HAProxy on is approaching its limits in terms of disk I/O
> due to our debug-level logging.
>
>
>
> The CPU is barely used, memory is no problem at all - the bottleneck will
> soon be the rate at which we can log to disk. Because the machine is more
> than adequate in all other areas, we'd really like to reduce the I/O load of
> logging.
>
>
>
> We are using the debug log output to tell us the response time and status
> code for various URLs - this is an essential task we can't do without.
>
>
>
> Is there any way to get this information without logging and
> post-processing every request? Can HAProxy dump out averaged statistics like
> this? Can we somehow reduce the I/O load by just logging the exact fields
> we're interested in?
>
>
>
> Many thanks!
>
> James
Received on 2009/02/12 02:38