Re: Rate Limiting Blog Link

Hey Bradford.

Have you considered doing this with iptables (Linux) or pf (BSD)? You'll want to use port 443 for your HTTPs problem, of course...

http://codingfreak.blogspot.com/2010/01/iptables-rate-limit-incoming.html

But be wary of NATs and such, especially if you have a high traffic site.  For example, our shop was hit by many connections at a university that triggered our alerts (just a scripted sort of IPs into the HAProxy front ends via Netstat) that we suspected a DDoS attack. Turns out we just got popular behind this NAT.

regards!
Carlo

On Tue, Apr 19, 2011 at 4:12 PM, bradford <fingermark#gmail.com> wrote:

> My whole concern is simplifying the rate limiting process and being able to
> have it work with https traffic (where it's not susceptible to spoofing).
> Can HAProxy do the latter without its own HTTPS implementation?
>
> Thanks for the tip and the post, Kyle.
>
> Bradford
>
>
> On Tue, Apr 19, 2011 at 11:38 AM, Kyle Brandt <kyle#stackoverflow.com>wrote:
>
>> Hi Bradford,
>>
>> To send to violators to a different backend, based of the example I used
>> in that post you want something like:
>>
>> In Frontend:
>> use_backend go-away if source_is_abuser
>>
>> Then a backend like:
>> backend go-away
>> mode http
>> errorfile 503 /etc/haproxy/errors/503rate.http
>>
>> Not sure off hand how to make it work with a reserved word however, hope
>> this helps.
>> -Kyle
>>
>> On Tue, Apr 12, 2011 at 9:37 AM, bradford <fingermark#gmail.com> wrote:
>>
>>> Excellent point, Jonathan. So, would having HAProxy support/implement
>>> HTTPS be the only way to allow HTTPS rate limiting (in HTTPS only and HTTP
>>> and HTTPS mixed environments)?
>>>
>>> As for my other point. Have you looked at the sample configuration on http://blog.serverfault.com/post/101649187use_backend
>>> go-away if source_is_abuser3/<http://blog.serverfault.com/post/1016491873/>
>>>
>>>
>>> It's a lot of configuration. And in that post it even describes part of
>>> the configuration as "more cryptic but is not too complicated." I don't
>>> know many people who could configure their server to do rate limiting
>>> without that blog post (and just the documentation). Moreover, if you took
>>> over a project and saw this configuration, it'd take you a bit to figure out
>>> what's going on. There are also statements in that post such as "the expire
>>> argument is how long to keep an entry in the table (In this case it just
>>> needs to be twice the length of the longest rate argument for a smoothed
>>> average). The time arguments for connection rate and bytes out rate are how
>>> long to calculate the average over."
>>>
>>> I just want a rate-limit reserved word that allows me to control
>>> connection rate / second (and bytes out rate), where i can send to some
>>> additional backend if violated.
>>>
>>>
>>> On Mon, Apr 11, 2011 at 5:47 AM, Jonathan Matthews <
>>> contact#jpluscplusm.com> wrote:
>>>
>>>> On 6 April 2011 16:42, bradford <fingermark#gmail.com> wrote:
>>>> > Also, in a previous email I mentioned something about
>>>> > X-Forwarded-For IP addresses being comma delimited. This table would
>>>> have
>>>> > to take that into consideration, I guess.
>>>>
>>>> No it shouldn't.
>>>> If you rate-limit based on information that you find in the XFF header
>>>> you allow malicious users to
>>>>
>>>> a) bypass the rate-limit by faking up different XFF headers each time or
>>>> b) DoS legitimate users by faking up the same, matching, XFF header
>>>> each time and letting haproxy do the DoS for them
>>>>
>>>> Also, above and beyond "I haven't understood it yet", the rest of your
>>>> email was rather light on *detail*. If other people are comprehending
>>>> and happily using the functionality based on the existing config
>>>> requirements and documentation, then perhaps the flaw doesn't lie with
>>>> the config and/or documentation.
>>>>
>>>> My 2-pence,
>>>> Jonathan
>>>> --
>>>> Jonathan Matthews
>>>> London, UK
>>>> http://www.jpluscplusm.com/contact.html
>>>>
>>>>
>>>
>>
>
Received on 2011/04/20 02:21