RE: NET or CPU VP Class : confusing NETTYPE recommendations

Stephen Faehn wrote: > Mario, > My company is running a SUN E4500 in our production environment as the DB > server/CI. > We are running SAP R/3 (3.1H). > The E4500 has 6 Processors and 4 gig memory. > We have 3 E3000 application servers connected to the CI. > > Here is what I have notice when moving poll threads off the CPU oninit > process, > to their own dedicated oninit, or NET VP process : > (1) From the output of `onstat -g sch`, semops/busy waits have gone up > significantly for > all the CPU process. If you run your a poll threads on a CPU vp then there are NO busy waits on this CPU vp. A busy wait is when the process spins attempting to avoid semop. If you have an inline poll thread (ie a poll thread running on the cpu VP) then the cpu vp will run the poll thread instead of spinning. The idea is that the next unit of work will come accross the network or be put on the ready queue. Now instead of the CPU VP spinning needlessly wasting time, it is polling waiting on I/O and if still does not find work to do, then it will semop. The other advantage of running poll threads on the CPU VP is that when a network message comes in the net vp must notifiy the CPU vp and force a context switch on busy system. Now if they are all running on the same CPU vp this is just a thread switch. > (2) From the output of `onstat -g glo`, syscpu times have gone down > significantly for > all the CPU process. The reason the system time has gone down is that the poll call is not being done on the CPU vp any more, but pushed off to the NETVP. By default the poll thread runs every 10 thread switch. So if you are doing 1000 threads switch a second then you will be doing 100 poll calls a second. > (3) Users haven't reported any increase/decrease in performance from SAP. > > I believe that the semops/busy waits went up because the CPU oninit process > have less work > to do now, as they don't have to "poll" for network connections. The just > spin, checking > for something to do. > This would also seem to agree with the syscpu times of the CPU process going > down. > So, I would say that what Art Kagel, Informix and SUN have said is the best > method to use. > (SHM connections to CPU VP's. tli connections to NET VP's). The best is to whenever possible run you poll threads on your CPU vps. I have seen a few rare cases when the operating system is using half a hardware processor and you do not want to run a CPU vp on that processor so you bind a NET VP there, but that is usually more for the really big boxes that are running really close to 95% busy or benchmarks. You need to run a net vp if you want to run two protocals on a single processor or there is an Informix bug that can be worked around by running the poll thread on the NET vp. Lastly if you are not totally bored, most if not all of the large SAP benchmarks run the poll threads on the CPU vps, not the NET vps. ---jmiller John Miller > I'm not sure what you mean by "NETTYPE settings in conjunction with > processor affinity". > Use processor affinity (AFF_SPROC AFF_NPROCS in onconfig). > Your CPU oninit's (as evident from `pbind`) will run on a dedicated > processor (good for cache) . > We use it here. > Use the NETTYPE parameter to put the tli poll threads on NET VP's, SHM on > CPU VP's. > Spread the anticipated number of sapr3 user threads from the application > servers > over the NET VP's. > > As you may have already noticed, SAP's OSS notes don't always seem to make > sense. > > -----Original Message----- > From: MOPSOMER@raychem.com [mailto:MOPSOMER@raychem.com] > Sent: Tuesday, April 20, 1999 4:52 PM > To: informix-list@iiug.org; Sean Kelsey > Subject: Re: NET or CPU VP Class : confusing NETTYPE recommendations > > Hi Sean, > > I am also puzzled by this. The recommendations for NETTYPE are confusing. > I > have read almost as many different recommendations as there are > possibilities. > > Art Kagel : TechNote Article 1998 - Volume 8 - Issue 3 : "Tuning Informix > Dynamic Server and Your System for Optimum Performance" : > - shared memory connections always on CPU VPs > - tli connections always on NET VPs > - use as many poll threads for shm as you have CPU VPs > > SUN Informix specialist : article "Tuning Informix for OLTP Workloads" > - for high transaction rate OLTP, use NET VPs for polling > - using CPU VPs causes continues switching from processing > queries to handling network operations (bad for processor cache) > - bind CPU VPs to physical processors (poll thread will use others) > > Informix 7.3 Performance Guide page 3-18 and following : > => not all that clear to me > > SAP R/3 OSS Note 41360 : > - CPU VPs for tli connections in case of different machines for > the SAP R/3 application instances > - NET VPs for shm connections in case there is little activity > (user connections) on the database server > > - Is it better to have poll threads running on CPU VPs or NET VPs ? > > - Is it better to have a few poll threads supporting many connections each > or a lot of poll threads ? > > - When choosing CPU VPs for the most used connections, will this interfer > with all other work that the CPU VPs already must handle (KAIO is used) ? > > - Informix books and course notes state that you can not use CPU VPs for > more than one protocol, but apparently it does work. We have been using > CPU VPs for shared memory connections and for TLI connections for a very > long time. Is it dangerous/inefficient to use CPU VPs for both protocols ? > > - Should I look at NETTYPE settings in conjunction with processor affinity > (setting the AFF_ ONCONFIG parameters or calling the pbind command) ? > > - Does anyone have suggestions for our environment ? We have a db server > (SUN) with 18 processors and a small SAP R/3 (central) instance on it and 8 > SAP R/3 application servers accessing that db server (tli) ? We have 50 > TLI connections per SAP application server (although this represents more > users - SAP uses "workprocesses", something similar to virtual processors). > > Kind regards, > Mario > > PS : We also had an instance crash recently. Informix recommended to stop > using shared memory connections, and use the safer TLI connections instead. > In our case, this was not a big problem as most of the work was already > using TLI connections, so the impact on global performance was minimal. > > ______________________________ Reply Separator > _________________________________ > Subject: NET or CPU VP Class > Author: "Sean Kelsey" <seank@teepee.demon.co.uk> at internet > Date: 20/4/99 11:19 > > Good morning Informixers. > > We have been going through a series of conversations with Informix > technical > support and I thought I would like to test all you DBA's out there as to > what the general consensus is regarding the VP class that the soctcp poll > thread should run on. > > We are using IDS 7.30.FC6 on HP-UX 11.x. We were

Mario, My company is running a SUN E4500 in our production environment as the DB server/CI. We are running SAP R/3 (3.1H). The E4500 has 6 Processors and 4 gig memory. We have 3 E3000 application servers connected to the CI. Here is what I have notice when moving poll threads off the CPU oninit process, to their own dedicated oninit, or NET VP process : (1) From the output of `onstat -g sch`, semops/busy waits have gone up significantly for all the CPU process. (2) From the output of `onstat -g glo`, syscpu times have gone down significantly for all the CPU process. (3) Users haven't reported any increase/decrease in performance from SAP. I believe that the semops/busy waits went up because the CPU oninit process have less work to do now, as they don't have to "poll" for network connections. The just spin, checking for something to do. This would also seem to agree with the syscpu times of the CPU process going down. So, I would say that what Art Kagel, Informix and SUN have said is the best method to use. (SHM connections to CPU VP's. tli connections to NET VP's). I'm not sure what you mean by "NETTYPE settings in conjunction with processor affinity". Use processor affinity (AFF_SPROC AFF_NPROCS in onconfig). Your CPU oninit's (as evident from `pbind`) will run on a dedicated processor (good for cache) . We use it here. Use the NETTYPE parameter to put the tli poll threads on NET VP's, SHM on CPU VP's. Spread the anticipated number of sapr3 user threads from the application servers over the NET VP's. As you may have already noticed, SAP's OSS notes don't always seem to make sense. -----Original Message----- From: MOPSOMER@raychem.com [mailto:MOPSOMER@raychem.com] Sent: Tuesday, April 20, 1999 4:52 PM To: informix-list@iiug.org; Sean Kelsey Subject: Re: NET or CPU VP Class : confusing NETTYPE recommendations Hi Sean, I am also puzzled by this. The recommendations for NETTYPE are confusing. I have read almost as many different recommendations as there are possibilities. Art Kagel : TechNote Article 1998 - Volume 8 - Issue 3 : "Tuning Informix Dynamic Server and Your System for Optimum Performance" : - shared memory connections always on CPU VPs - tli connections always on NET VPs - use as many poll threads for shm as you have CPU VPs SUN Informix specialist : article "Tuning Informix for OLTP Workloads" - for high transaction rate OLTP, use NET VPs for polling - using CPU VPs causes continues switching from processing queries to handling network operations (bad for processor cache) - bind CPU VPs to physical processors (poll thread will use others) Informix 7.3 Performance Guide page 3-18 and following : => not all that clear to me SAP R/3 OSS Note 41360 : - CPU VPs for tli connections in case of different machines for the SAP R/3 application instances - NET VPs for shm connections in case there is little activity (user connections) on the database server - Is it better to have poll threads running on CPU VPs or NET VPs ? - Is it better to have a few poll threads supporting many connections each or a lot of poll threads ? - When choosing CPU VPs for the most used connections, will this interfer with all other work that the CPU VPs already must handle (KAIO is used) ? - Informix books and course notes state that you can not use CPU VPs for more than one protocol, but apparently it does work. We have been using CPU VPs for shared memory connections and for TLI connections for a very long time. Is it dangerous/inefficient to use CPU VPs for both protocols ? - Should I look at NETTYPE settings in conjunction with processor affinity (setting the AFF_ ONCONFIG parameters or calling the pbind command) ? - Does anyone have suggestions for our environment ? We have a db server (SUN) with 18 processors and a small SAP R/3 (central) instance on it and 8 SAP R/3 application servers accessing that db server (tli) ? We have 50 TLI connections per SAP application server (although this represents more users - SAP uses "workprocesses", something similar to virtual processors). Kind regards, Mario PS : We also had an instance crash recently. Informix recommended to stop using shared memory connections, and use the safer TLI connections instead. In our case, this was not a big problem as most of the work was already using TLI connections, so the impact on global performance was minimal. ______________________________ Reply Separator _________________________________ Subject: NET or CPU VP Class Author: "Sean Kelsey" <seank@teepee.demon.co.uk> at internet Date: 20/4/99 11:19 Good morning Informixers. We have been going through a series of conversations with Informix technical support and I thought I would like to test all you DBA's out there as to what the general consensus is regarding the VP class that the soctcp poll thread should run on. We are using IDS 7.30.FC6 on HP-UX 11.x. We were having problems with earlier releases of the engine ( 7.24.UC7 ) where we were getting Assert Failures due to the soctcppoll thread crashing. Informix Tech Support told us that it was a know bug and we should adopt to one of the following:- 1. Reduce the number of poll threads that run 2. Increase the number of users that each poll thread services 3. change the class of VP from CPU to NET ( 4. Upgrade, of course ) As you can see from the below NETTYPE settings, we opted for a mixture of 1 & 2 but we are occasionally getting the same assert failure. NETTYPE ipcshm,1,50,NET NETTYPE soctcp,2,300,CPU # This is the new one # NETTYPE soctcp,3,200,CPU # This was the old setting NETTYPE sqlmux,,, So, okay, if we change the VP class to NET, will this not slow things down significantly for all user connections ? Can anyone point out what benefits (if any) changing the VP class will give us and which ones do you recommend... TIA as always Sean