Request for help

Hi, A request for help here! We're currently performing a rearchitect of our system and have a major architectural decision to take on a database issue, so any wisdom (or pointers to wisdom in the form of papers etc.) which anyone on the net has would be greatly appreciated. Firstly, a bit of background about the target hardware platform which we will be running on: our system will be running on a network of HP 9000s (800 series) machines; our application will run on one HP with the other running as an Informix database server and running other software. In our current architecture, we have a single UNIX process which consists of a framework executing a number of state machines. The unit of scheduling of these state machines is a state transition. Most of our state transitions involve database accesses (reads and or writes). Within our current system, we only have a small database and our database is on the same machine as our application software. Even so, it appears that synchronous database accesses are a bottleneck in the system since each such access will have quite a large dead time (due to our scheduling mechanism). As our database grows in size (and the number of concurrent executing state machines is proportional to the database size), this becomes a much greater problem; especially since a smaller and smaller percentage of our database will be held within the cache. The system must respond in real time; there are certain conditions under which a large number of state machines must execute concurrently. We have to address this problem, and to do so have come up with a number of potential solutions:- 1) Rewrite the scheduler - this is not really an option. 2) Implement each state machine as a separate UNIX process - a lot of the state machines are quite small: thus the overhead of spawning off the processes required every time may negate any savings made. In addition, this may result in a very large number of UNIX processes running which will affect the relative performance of the rest of our system (as well as our database server). 3) Develop a server process which sits between our application process and Informix and which handles asynchronous database requests. State transitions within our state machine would then cease to be dependent on a synchronous data communication across a network - thus other state machines would be free to run. Advantages i) Considerable performance improvement in busy system ii) Number of server processes tunable to help further Disadvantages i) Greater complexity in our software (twice as many states) ii) Small performance hit in (comparitively) idle system. 4) Have a number of application processes all running; share state machines amongst them on demand. Advantages i) Requires lesser complexity in our software (keep same number of states) ii) Number of application processes tunable Disadvantages i) Performance improvement likely to be less dramatic in busy system; ii) COnsiderable complexity likely in state machine load balancing algorithm. 5) Buy a faster machine!!! - unlikely to be a suitable solution as we seem likely to be I/O bound and LAN bound which we cannot do a great deal about! There may be other solutions which we have not thought about. We are worried that we may take a decision about our architecture without having sufficient knowledge of the problems which we may encounter in adopting one of the solutions. If anybody has come up against similar problems and has any insights which they could share I would be very greatful. Thanks Dave David Gourley Telephone: +44-666 832557 AT&T Network Systems, Swindon Road, Malmesbury, Wiltshire, England. email: david.gourley@att.com (Internet) att!mlgtw!mltsa!dgourley (UUCP)