[NTLUG:Discuss] Distributed processing

[Greg Edwards]

cbbrowne at hex.net wrote:
> 
> On Wed, 18 Jul 2001 21:34:28 CDT, the world broke into rejoicing as
> Greg Edwards <greg at nas-inet.com>  said:
> > Each host would be assigned a set of functions that it would support and
> > not a set of applications.  This does not fit the load balance or
> > clustering model very well.  Some hosts could act as database servers,
> > some in a Beowulf cluster, some as game tree processors, some as search
> > engines (these are just examples).  Not every host would support every
> > request that could be generated.  This makes for a poor fit for a strict
> > message queue solution as you don't want to send the queue to hosts that
> > are non players in the logic thread.
> 
> I would object _slightly_ on the message queue comment; it would be quite
> reasonable to have a set of queues named suitably for better sharing...
> I'm not sure what your exact objection is here, but would speculate it's
> of two things:
> 


> 
> Sounds fairly suited for the message queueing approach, albeit not as
> an automatic result of the messaging scheme, but rather as an aspect
> of the design of the work process.
> --


A message queue solution would certainly work but it would not be
efficient for what I'm looking at.  I envision several hundred functions
distributed throughout the farm with many functions capable of retaining
results for an aged period.  Trying to maintain response factor
heuristics with message queues would (IMO) flood the backbone with more
traffic than is necessary.  I believe that maintaining heuristics on the
response from the current host to the host that processed the request
using an open hash would be simpler to maintain and a considerably
lighter load on the network.

Consider a multiplier game spanning a game board with 64 grids and 15
players currently in the game.  Host A is currently the game manager. 
Hosts C and D are responsible for determining which players are visible
to each other.  Players 1, 5, 9, and 10 are each in grid 32.  Hosts C
and D currently have the same response factor (as known to Host A). 
Player 1 gets processed by host C and player 5 by host D.  All other
things being equal host C and D provide equal response factors on the
returned results.  Now player 9 comes along and host C gets the
request.  Since host C and D currently both have the search results from
the 1 and 5 players cached the performance factor of host C becomes
better than D (using simple averaging).  Now when player 10 comes along
host C has already had player 9 results added to its data set and host D
has not.  In this scenario you want host C to get the request and not
host D.  In a typical message queue solution (first come first serve)
host D has a better than average chance of ending up with the request. 
Yes we could get into weighting the queues by host response factors, but
would that be worth the work.

For those thinking "hey this sounds like an OOPS solution".  I want it
to be a procedural solution for process control reasons.  I want to be
able to backup and request from a different host if I receive a
timeout/non-response on any requests.  I also do NOT want to pass the
entire data set between functions (hosts).

-- 
Greg Edwards
New Age Software, Inc.
http://www.nas-inet.com