Hello,
An other point that could be enhanced in the log format is the date format: we miss the timezone in IT
Best regards,
-- Damien 2011/11/29 Damien Hardy <damienhardy.bal#gmail.com>Received on 2011/11/30 10:50
> Hello haproxy-list,
>
> We are working on near real time web statistics architecture based on the
> haproxy access logs.
> (a first shoot of a flume plugin is available here :
> https://github.com/figarocms/flume-haproxy-extractor it is now designed
> to feet or needs especially but we are working on making it available for
> more general pupose)
>
> But HTTP logs are quite difficult to parse especially if there is captured
> headers:
> * Request and response headers could be captured enclosed both by {}, but
> if you capture only one of them in a frontend there is no way to now if it
> is the request or the response header. because if the capture is not set
> the field is not set either in teh log.
> My suggestion is maybe to add a letter before "{" to precise if it's query
> ( ex : "Q{" ) or response ( "R{" ) set of captured headers. or always make
> them appear.
>
> * Olso the header captured is not precised in it, only values appear so
> you have olso to know the order of capture in the configuration to parse it
> well.
> Maybe the whole line of the header could be set in the field.
>
> * The headers are separated by a "|" but the eventuelly "|" allready
> present in the headers are not escaped, that could occure some probleme of
> parsing I suppose.
> No really answer to address this probleme but maybe enclosing headers by "
> (and escaped this char it like many other String method could do) could be
> a good envolvment.
>
> And As I precised it before syslog default size of 1024 is very short for
> today statistical need. I had to increase the constant to 4096 (capture of
> full UA, and Referer are costly).
>
> Best regards,
>
> --
> Damien
>
>
>
>
>
>
>
>
>
>
This archive was generated by hypermail 2.2.0 : 2011/11/30 11:00 CET