haproxy for multiple backend requests for single resource

Hi all,

  We've been happily using haproxy as a load balancer between a pool of apache boxes and a pool or tomcat boxes for 3 years in AWS. Amazing piece of software that gives us zero issues.

  Question. Has anyone made/altered haproxy to the following:

1. take an incoming request for a resource
2. farm that request out to N backend servers - yes the same request to N servers
3. Assemble the N responses and push them back up to the requester

  There are several scenarios where this seems interesting. One is implementing a distributed reads to a backend pool of key-value stores, where the requester wants the N results (minus the ones that timeout) and will decide which one is most recent version. (assume a consistent hashing alg is used to spread the writes across the pool of N kv stores).

  The second is a general variant of the first one. Distributed multi-request of any HTTP resource with same path where each of the N servers in the pool may have a different response, it's up to the requester to decide what to do with the N responses.

  Assume the requester is smart enough to deal with concatenated HTTP responses that would come back from the multi-proxy process.

  I have a strong hunch that harpoxy's event driven architecture and solid connection handling will support this better than the naive approach of starting N threads to each made a different HTTP request. ;-)

Thanks - Neal Richter
Bozeman, Montana Received on 2010/02/10 05:09