Many ajax connections and keep-alive

  Hi.

We have the following task:

The webbrowser gets a list of 20 to 100 items to be processed. Currently we have 5 processing servers, each with 4 cores. This makes a optinal processing capacaty of 4x5=20 at one time. Each processing server processes one item at a time. The processing of an item can take from 2 seconds to 2 minutes. The arcitecture need to be able to scale up to handle many users at once.

My suggestion to solve the task using HAProxy:

• Webrowser open up 10 ajax calls at a time.
• Each ajax call is processed by haproxy and an available processing server is assigned, or put in queue.
• When a item is processed, the result is displayed on page using Javascript, and another ajax call is opened until all items has been processed.
• Keep-alive is enabled to reduce the number of connections to HAProxy.

The developement team however thinks this is the better solution:

• Browser opens a single ajax call to a processing manger server with the entire list of items.
• The Processing manager connects to the processing servers and combines the results.
• Each 20 seconds a new ajax call is opened to the processing manager to return any finished items and the result is shown on the page.

The development team thinks that creating up to 100 ajax connections per user will result in too much overhead. My plan to solve this is by only opening 10 ajax calls at one time and use the browsers keep-alive to solve the problem.

Would keep-alive work so that only 10 connections will be opened by the browser?

Any better solution to this task using haproxy?

-- 
mvh.

Daniel Storjordet

D E S T ! N O :: Strandgata 117 :: 4307 Sandnes
Mob 45 51 73 71 :: Tel 51 62 50 14  daniel@desti.no :: http://www.desti.no
www.destinet.no - Webpublisering på nett
www.func.no - Flysøk på nett