traffic still going to backends marked as DOWN

We have the following setup:

nginx -> haproxy (1.3.15.3) -> mongrels

Config is here:

http://pastie.org/270874

Problem: after I start haproxy and then immediately go to the web stats it will start checking each backend in turn and when it gets to backend 6 and 8 mark them as down. However, in looking at the logs traffic is STILL being sent to those mongrels and they are serving it up successfully.

How can this be the case? From my understanding, if haproxy marks a server as down then it will never send traffic to it. Yet, traffic is being sent (and successfully handled) to that backend...

Is there something wrong with my config? In this case, it seems a couple of things are going that are odd to me:

1. Why is haproxy marking it as down
2. If haproxy does consider it down (for whatever reasons), why is traffic still being sent to it

Now, notice I have chosen to use a TCP check and not an HTTP check (since we have other background monitoring agents to actually restart a downed service). If I move to an HTTP check that *might* solve question #1, but doesnt answer #2.

Thanks in advance

• Rupert