Re: NETTYPE NET/CPU

On Sep 17, 12:37 pm, "Art S. Kagel" <art.ka...@gmail.com> wrote: > On Sep 17, 12:59 pm, mohitanch...@gmail.com wrote: > <SNIP> > > > I am posting it again as this is a separate question: > > > But when NETTYPE is set to NET it spawns different set of oninit, > > which is additional set of oninit as compared to the one started by > >informixfor NUMCPUVPS. So why is there a separate parameter > > NUMCPUVPS> > and why there is no such parameter for NET, something like > > NUMNETVPS. > > I can feel there is some difference for not having something like > > this, but I am not exactly sure. > > OK, now I get what you don't get. NUMCPUVPS (the original pre-9.21 > parameter) and VPCLASS...CPU settings configure the number of virtual > processors (read oninit processES) that are running. This number is > NOT affected by ANY setting of the NETTYPE parameter. Indeed if you > configure more poll threads assigned to CPU VPs than are configured in > NUMCPUVPS / VPCLASS the engine will simply launch NET VPs to handle > the excess over and above NUMCPUVPS. You may ask "But these are all > copies of oninit so aren't they the same?". The answer is "No, they > are not." Once a copy of oninit is assigned to a VP class (ie: CPU, > NET, ADM, MSC, AIO, USER, or whatever) it behaves differently from > oninits assigned to a different VP class and performs different > functions. Only a CPU VP will perform queries against the data, sort > data streams, schedule outstanding requests, etc. Only AIO VPs > perform IOs against COOKED files, etc. > > Now, why have two classes of poll thread? One, the NET class, > operates in a separate set of VPs (oninit instance) while the other, > the CPU class, operates as part of one or more of the CPU VP > instances. > > Why didInformix(now IBM) bother to have two classes of poll > threads? The thinking was so that the users (ie DBAs) could tune how > the engine polled to adjust responsiveness. Informixoriginally > recommended that CPU VPs handle network connection polling under the > probable assumption that most user connections would be via network > and having the CPU VP pick up the requests directly and schedule them > immediately was best for throughput. The NET VPs were intended for > when the number of connections exceeded what the CPU VPs could handle > and for shared memory connections which were envisioned to be mostly > for administering the engine. > > I REALLY didn't want to go into this again, and feel free to ignore > the rest, but... > > The problem with this original thinking is that the CPU VPs cannot > block on the network ports to listen for new requests and new > connections, they have to process existing requests also (their main > job I might say). So, the CPU VP has to poll each port periodically > when it's not too busy processing existing work. That means that the > engine is actually less responsive than it might be. Meanwhile, if > the shared memory connection poll threads are set up in NET VPs, which > have nothing better to do, they cannot block on a shared memory read > there's no mechanism for that, so they have to poll all configured > shared memory connection structures constantly looking for new > requests and connections. This burns CPU cycles that the other VPS > could be using for processing and involves the OS kernel thousands of > system calls per second to latch and release semaphores or global > mutexes. Also bad. Similarly, the CPU VPs polling the ports also > involves loads of system calls. More bad. > > Reviewing this logic, which explained why our own systems here were > being inundated with system calls as soon as theInformixengines > started up, and why the the system call overhead was worse when the > systems were idle than when they were running near capacity, I > reasoned that perhaps I should try it the other way around and that > worked. NET VPs listening on the TCP ports can block until there is > data to read and the CPU VP will poll the shared memory connections > far less often then hte NET VPs did. Voila, system call storms > disappeared. > > Then I noticed that the one CPU VP that had the shared memory poll > thread running in it, VP #1, was using 80% of the total CPU time that > all of the 10 CPU VPs were using and the second most busy VP was using > only 10% of the total time. That wasn't taking good advantage of the > many processors on the box but was inundating one CPU. So I tried > setting NETTYPE ipcshm to the same number of listeners as there were > CPU VPs in NUMCPUVPS. Poof! Now there was a smooth curve with the > CPU usage spread in a smooth decline from the first VP using about 25% > on down. The system became suddenly much more responsive. Query > times were more consistent because wait time before being placed into > the ready queue was way down. > > Art S. Kagel This explains a lot. We also saw similar gains when we switched to NET VP for soctcp connections. This brings me to my next question about KAIO VP. 1. Why are there 2 types of AIO VP. 2. Which one is more efficient 3. Looks like user can't control KAIO VP. So I am not sure how many Informix will start and what determines this. 4. Also, are AIO VPs used when KAIO VPs are being run by Informix.

On 21/09/2007, mohitanchlia@gmail.com <mohitanchlia@gmail.com> wrote: > On Sep 17, 12:37 pm, "Art S. Kagel" <art.ka...@gmail.com> wrote: > > On Sep 17, 12:59 pm, mohitanch...@gmail.com wrote: > > <SNIP> > > > > > I am posting it again as this is a separate question: > > > > > But when NETTYPE is set to NET it spawns different set of oninit, > > > which is additional set of oninit as compared to the one started by > > >informixfor NUMCPUVPS. So why is there a separate parameter > > > NUMCPUVPS> > > and why there is no such parameter for NET, something like > > > NUMNETVPS. > > > I can feel there is some difference for not having something like > > > this, but I am not exactly sure. > > > > OK, now I get what you don't get. NUMCPUVPS (the original pre-9.21 > > parameter) and VPCLASS...CPU settings configure the number of virtual > > processors (read oninit processES) that are running. This number is > > NOT affected by ANY setting of the NETTYPE parameter. Indeed if you > > configure more poll threads assigned to CPU VPs than are configured in > > NUMCPUVPS / VPCLASS the engine will simply launch NET VPs to handle > > the excess over and above NUMCPUVPS. You may ask "But these are all > > copies of oninit so aren't they the same?". The answer is "No, they > > are not." Once a copy of oninit is assigned to a VP class (ie: CPU, > > NET, ADM, MSC, AIO, USER, or whatever) it behaves differently from > > oninits assigned to a different VP class and performs different > > functions. Only a CPU VP will perform queries against the data, sort > > data streams, schedule outstanding requests, etc. Only AIO VPs > > perform IOs against COOKED files, etc. > > > > Now, why have two classes of poll thread? One, the NET class, > > operates in a separate set of VPs (oninit instance) while the other, > > the CPU class, operates as part of one or more of the CPU VP > > instances. > > > > Why didInformix(now IBM) bother to have two classes of poll > > threads? The thinking was so that the users (ie DBAs) could tune how > > the engine polled to adjust responsiveness. Informixoriginally > > recommended that CPU VPs handle network connection polling under the > > probable assumption that most user connections would be via network > > and having the CPU VP pick up the requests directly and schedule them > > immediately was best for throughput. The NET VPs were intended for > > when the number of connections exceeded what the CPU VPs could handle > > and for shared memory connections which were envisioned to be mostly > > for administering the engine. > > > > I REALLY didn't want to go into this again, and feel free to ignore > > the rest, but... > > > > The problem with this original thinking is that the CPU VPs cannot > > block on the network ports to listen for new requests and new > > connections, they have to process existing requests also (their main > > job I might say). So, the CPU VP has to poll each port periodically > > when it's not too busy processing existing work. That means that the > > engine is actually less responsive than it might be. Meanwhile, if > > the shared memory connection poll threads are set up in NET VPs, which > > have nothing better to do, they cannot block on a shared memory read > > there's no mechanism for that, so they have to poll all configured > > shared memory connection structures constantly looking for new > > requests and connections. This burns CPU cycles that the other VPS > > could be using for processing and involves the OS kernel thousands of > > system calls per second to latch and release semaphores or global > > mutexes. Also bad. Similarly, the CPU VPs polling the ports also > > involves loads of system calls. More bad. > > > > Reviewing this logic, which explained why our own systems here were > > being inundated with system calls as soon as theInformixengines > > started up, and why the the system call overhead was worse when the > > systems were idle than when they were running near capacity, I > > reasoned that perhaps I should try it the other way around and that > > worked. NET VPs listening on the TCP ports can block until there is > > data to read and the CPU VP will poll the shared memory connections > > far less often then hte NET VPs did. Voila, system call storms > > disappeared. > > > > Then I noticed that the one CPU VP that had the shared memory poll > > thread running in it, VP #1, was using 80% of the total CPU time that > > all of the 10 CPU VPs were using and the second most busy VP was using > > only 10% of the total time. That wasn't taking good advantage of the > > many processors on the box but was inundating one CPU. So I tried > > setting NETTYPE ipcshm to the same number of listeners as there were > > CPU VPs in NUMCPUVPS. Poof! Now there was a smooth curve with the > > CPU usage spread in a smooth decline from the first VP using about 25% > > on down. The system became suddenly much more responsive. Query > > times were more consistent because wait time before being placed into > > the ready queue was way down. > > > > Art S. Kagel > > This explains a lot. We also saw similar gains when we switched to NET > VP for soctcp connections. This brings me to my next question about > KAIO VP. > 1. Why are there 2 types of AIO VP. > 2. Which one is more efficient > 3. Looks like user can't control KAIO VP. So I am not sure how many > Informix will start and what determines this. > 4. Also, are AIO VPs used when KAIO VPs are being run by Informix. > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list > AIO is the mechanism used by IDS to write to cooked files and file-systems specificaly and raw file store is KAIO is not available. Even if you are using KAIO there must be 4 AIO VPs configured to handle writing logical logs to tape/disk, writing console and error logs etc. KAIO is determined by the O/S, not all are capable. If they are then the configuration is achived within the kernel andmonitoring is at O/S level. It is the most efficient way of writing to disk as it takes load off the engine and bypasses the maximum number of steps in writing to disk. Keith