Hypermail

From: Willy Tarreau <w#1wt.eu>
Date: Thu, 18 Oct 2007 23:45:35 +0200

Hi all!

I've just released HAProxy 1.3.13. It's been a very long time, but quite a bunch of things have moved inside it, without real apparent impact for the user.

The most important things (to my eyes) is the new protocol stack which is being built. Right now it is possible to create transport protocols with I/O functions, and to assign listeners to them. It's a big step forwards in the direction of the support for a richer protocol set. Now, when you declare a listener, it belongs to a protocol and can have its own accept() function without being tied to a frontend. It will make it easier to later implement support for FTP...

Based on this, I have implemented support for client-side UNIX domain sockets (SOCK_STREAM only). It has allowed me to create trivial entries in the global section to bind to a UNIX socket without the need of any listener. Some of you will see where I'm going... It is now possible to consult the overall statistics by simply connecting to a UNIX socket an writing "show stat" followed by a line feed :

$ printf "show stat\n" | socat unix-connect:/var/run/haproxy.stat stdio

# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,weight,act,bck,chkfail,chkdown
fast1,127.0.0.010,0,0,0,0,-,0,0,0,,0,,0,0,DOWN,10,1,0,0,1,
fast1,BACKEND,0,0,0,0,0,0,0,0,0,0,,0,0,DOWN,0,0,0,,,
slow1,127.0.0.001,0,0,0,0,-,0,0,0,,0,,0,0,UP,10,1,0,0,0,
slow1,127.0.0.002,0,0,0,0,-,0,0,0,,0,,0,0,UP,10,1,0,0,0,
slow1,127.0.0.003,0,0,0,0,-,0,0,0,,0,,0,0,UP,10,1,0,0,0,
slow1,127.0.0.004,0,0,0,0,-,0,0,0,,0,,0,0,UP,10,1,0,0,0,
slow1,127.0.0.005,0,0,0,0,-,0,0,0,,0,,0,0,UP,10,1,0,0,0,
slow1,BACKEND,0,0,0,0,0,0,0,0,0,0,,0,0,UP,50,5,0,,,

Stats_fnt,FRONTEND,,,0,1,100,51,21564,199002,0,0,0,,,OPEN,,,,,, Stats_fnt,BACKEND,0,0,0,0,100,0,21564,199002,0,0,,0,0,UP,0,0,0,,, $

To achieve this, you just need the following line in the global section :

global

stats socket /var/run/haproxy.stat mode 600

Check the configuration manual, there are a few keywords associated with the "stats" keyword.

Since I needed to output the UNIX stats in CSV format, it was a good opportunity to rework the ugly statistics dump subsystem. Some code has moved from proto_http.c to new protocol-independant files, and HTTP now supports the CSV output too (just add ";csv") to the request. The HTTP CSV mode is really suited for SNMP monitoring as it consumes very low resources: on my notebook at 1.7 GHz, I can consult the stats page 10000 times/s with 50% CPU for 5 lines of output data. So even if you want to poll every second for large configs, it should remain unnoticed.

Speaking of the stats page, there have also been two minor changes from Krzysztof Oledzki :

count the servers' retries and backend's retries/redispatches
report haproxy's version by default. It can be hidden per backend using the "stats hide-version" keyword, or globally at build time by defining the macro STATS_VERSION_STRING to the empty string "". Finally, it is useful. My stats page at home reports to me that I'm running version 1.3.13, but you don't see it on the public side :-)

I have merged some trivial code I was using in parallel branches for a long time : the use of dlmalloc (Doug Lea's malloc). For those who don't happen to know it, it's a very nice malloc() library which relies on a good number of methods to allocate and free memory. And what I like in it is that it can entirely rely on mmap(). This is very convenient, because with mmap(), when you free one page, it is restituted to the system. (with malloc, as long as the highest byte is not free, nothing can be restituted). Combining dlmalloc with an automatic garbage collector is excellent during soft stop, because the memory usage fades out while the new process slowly consumes memory. Globally, you can have 10 processes on a system eating almost not more than one single process, because only one is active and the 9 others are terminating. Check the Makefile to know how to build with dlmalloc, it's quite easy.

Last but not least, Arnaud Cornet has worked on replacing the old O'reilly's code for which the license was not very clear in the past (MIT now), with some new code under LGPL. After several iterations, the code looked OK, so I merged it. Arnaud maintains the Debian port of haproxy and already uses this code in it. It does not mean it's bug-free, but that it has already been tested a little bit. The replaced code was only used in appsessions, so if you experience problems and are not using appsessions, it's most probably due to another bug I introduced.

All the rest is the same as what it in 1.3.12.3. Overall, I would expect the bug rate to be rather low in this version, because although a lot of code has moved, it was mainly on focused features.

For the next releases, I want to transform the current TCP listeners to use the new mechanism. I also plan to implement the weighted least-conn algorithm, a dynamic weighted round robin algo I've been designing last weeks (servers can change their weights at any moment), and perhaps also to replace the scheduler with a lighter one.

I'll still maintain 1.3.12.X in case horrible bugs surface, but will focus on newer release anyway. So the most reasonable deal would be for me to switch maintenance to 1.3.13.X (with 1.3.12.X in parallel), so that users still have one known working version, without preventing us from progressing on features and optimizations.

Just as for 1.3.12.3, the code is available at the usual place :

http://haproxy.1wt.eu/download/1.3/src/

Pre-compiled binaries for Linux/x86 and Solaris/ultrasparc are also available in the bin/ directory.

Regards,
Willy Received on 2007/10/18 23:45

[ANNOUNCE] haproxy 1.3.13