Single cpu vp woe

9.20 UC3 on Red Hat 7. This site is running the time series datablade in production on a 2-cpu box. It's all working fine. For reasons we needn't go into they want to move to a single cpu box. They've set up an identical test server. When we run the same programs with the live setting (NUMCPUVPS=2) we get the same consistent results. However, if we reduce NUMCPUVPS to 1 then (and regardless of whether there are 1 or 2 physical cpus present in the test box) the tests run 15-20 times more slowly. The reason isn't difficult to see: onstat -g rea shows the ready queue with 3 or more entries, indicating a bottleneck on cpu vps. The one test I didn't try, through lack of time, is running with MULTIPROCESSOR=1, NUMCPUVPS=2 with a single physical cpu. Anyway, I infer from all this that they need to run this version of IDS and the TimeSeries datablade on a 2-way box. But the customer isn't happy with this, and wants a decent and plausible explanation as to why this cannot be made to work on a single cpu server. I don't know enought about the datablade to give a more detailed opinion. Any comment gratefully received, as ever. thanks Neil

IDS uses preemptive scheduling, the OS uses preemptive with timeslice. Sometimes it's best to let the CPUVPS 'duke it out' by over configuring a bit so you get a bit of timeslice scheduling. ;-) My guess is that there are other things causing the hit. For instance, if a thread is a bit of a hog, then it could end up doing things like flushing the buffer pool because you're running with 1 cpuvp. In that case if you have two transactions running that are both hogs, then they both flush the buffer pool, but do it serially. So you would be able to get better performance by trying to run them in parallel. At least that way you might still have a page in the buffer pool that can be used by the other there --- sorta buddy bunch... M.P. "Neil Truby" <neil.truby@ardenta.com> wrote in message news:37k8t6F5fm45bU1@individual.net... > 9.20 UC3 on Red Hat 7. > > This site is running the time series datablade in production on a 2-cpu box. > It's all working fine. > > For reasons we needn't go into they want to move to a single cpu box. > They've set up an identical test server. When we run the same programs with > the live setting (NUMCPUVPS=2) we get the same consistent results. However, > if we reduce NUMCPUVPS to 1 then (and regardless of whether there are 1 or 2 > physical cpus present in the test box) the tests run 15-20 times more > slowly. The reason isn't difficult to see: onstat -g rea shows the ready > queue with 3 or more entries, indicating a bottleneck on cpu vps. > > The one test I didn't try, through lack of time, is running with > MULTIPROCESSOR=1, NUMCPUVPS=2 with a single physical cpu. > > Anyway, I infer from all this that they need to run this version of IDS and > the TimeSeries datablade on a 2-way box. But the customer isn't happy with > this, and wants a decent and plausible explanation as to why this cannot be > made to work on a single cpu server. I don't know enought about the > datablade to give a more detailed opinion. > > Any comment gratefully received, as ever. > > thanks > Neil > >

ARGGGGGGGGHHH!!!!! IDS uses non-preemptive scheduling. The OS uses preemptive (i.e. timeslice)... M.P. "Madison Pruet" <mpruet@comcast.net> wrote in message news:KcOdnTEoYtadYYnfRVn-vw@comcast.com... > IDS uses preemptive scheduling, the OS uses preemptive with timeslice. > Sometimes it's best to let the CPUVPS 'duke it out' by over configuring a > bit so you get a bit of timeslice scheduling. ;-) > > My guess is that there are other things causing the hit. For instance, if a > thread is a bit of a hog, then it could end up doing things like flushing > the buffer pool because you're running with 1 cpuvp. In that case if you > have two transactions running that are both hogs, then they both flush the > buffer pool, but do it serially. So you would be able to get better > performance by trying to run them in parallel. At least that way you might > still have a page in the buffer pool that can be used by the other there --- > sorta buddy bunch... > > > M.P. > > > "Neil Truby" <neil.truby@ardenta.com> wrote in message > news:37k8t6F5fm45bU1@individual.net... > > 9.20 UC3 on Red Hat 7. > > > > This site is running the time series datablade in production on a 2-cpu > box. > > It's all working fine. > > > > For reasons we needn't go into they want to move to a single cpu box. > > They've set up an identical test server. When we run the same programs > with > > the live setting (NUMCPUVPS=2) we get the same consistent results. > However, > > if we reduce NUMCPUVPS to 1 then (and regardless of whether there are 1 or > 2 > > physical cpus present in the test box) the tests run 15-20 times more > > slowly. The reason isn't difficult to see: onstat -g rea shows the ready > > queue with 3 or more entries, indicating a bottleneck on cpu vps. > > > > The one test I didn't try, through lack of time, is running with > > MULTIPROCESSOR=1, NUMCPUVPS=2 with a single physical cpu. > > > > Anyway, I infer from all this that they need to run this version of IDS > and > > the TimeSeries datablade on a 2-way box. But the customer isn't happy > with > > this, and wants a decent and plausible explanation as to why this cannot > be > > made to work on a single cpu server. I don't know enought about the > > datablade to give a more detailed opinion. > > > > Any comment gratefully received, as ever. > > > > thanks > > Neil > > > > > >

It was 9.21 UC3 btw, not 9.20. Are you saying that you can give a plausible explanation of why it will never work on a 1-cpu box? "Madison Pruet" <mpruet@comcast.net> wrote in message news:3MSdnYl8fIYXYYnfRVn-gg@comcast.com... > ARGGGGGGGGHHH!!!!! > > IDS uses non-preemptive scheduling. The OS uses preemptive (i.e. > timeslice)... > > M.P. > "Madison Pruet" <mpruet@comcast.net> wrote in message > news:KcOdnTEoYtadYYnfRVn-vw@comcast.com... >> IDS uses preemptive scheduling, the OS uses preemptive with timeslice. >> Sometimes it's best to let the CPUVPS 'duke it out' by over configuring a >> bit so you get a bit of timeslice scheduling. ;-) >> >> My guess is that there are other things causing the hit. For instance, >> if > a >> thread is a bit of a hog, then it could end up doing things like flushing >> the buffer pool because you're running with 1 cpuvp. In that case if you >> have two transactions running that are both hogs, then they both flush >> the >> buffer pool, but do it serially. So you would be able to get better >> performance by trying to run them in parallel. At least that way you > might >> still have a page in the buffer pool that can be used by the other > there --- >> sorta buddy bunch... >> >> >> M.P. >> >> >> "Neil Truby" <neil.truby@ardenta.com> wrote in message >> news:37k8t6F5fm45bU1@individual.net... >> > 9.20 UC3 on Red Hat 7. >> > >> > This site is running the time series datablade in production on a 2-cpu >> box. >> > It's all working fine. >> > >> > For reasons we needn't go into they want to move to a single cpu box. >> > They've set up an identical test server. When we run the same programs >> with >> > the live setting (NUMCPUVPS=2) we get the same consistent results. >> However, >> > if we reduce NUMCPUVPS to 1 then (and regardless of whether there are 1 > or >> 2 >> > physical cpus present in the test box) the tests run 15-20 times more >> > slowly. The reason isn't difficult to see: onstat -g rea shows the > ready >> > queue with 3 or more entries, indicating a bottleneck on cpu vps. >> > >> > The one test I didn't try, through lack of time, is running with >> > MULTIPROCESSOR=1, NUMCPUVPS=2 with a single physical cpu. >> > >> > Anyway, I infer from all this that they need to run this version of IDS >> and >> > the TimeSeries datablade on a 2-way box. But the customer isn't happy >> with >> > this, and wants a decent and plausible explanation as to why this >> > cannot >> be >> > made to work on a single cpu server. I don't know enought about the >> > datablade to give a more detailed opinion. >> > >> > Any comment gratefully received, as ever. >> > >> > thanks >> > Neil >> > >> > >> >> > >

A theoretical 'for instance' ---- Suppose that the application is such that it's causing a table scan on a fairly large table. Let's suppose also that the scan is a 'negative' scan which doesn't return data to the client until the total scan is done. Maybe an aggregate process, or has a where clause that is highly unlikly to return a match. The scan could cause a complete flush of the buffer pool before it found anything. --( N.B. all --- in such a case, the engine should be doing what we call 'nice yields...'). Now supposed that we want multiple of these applications running. With a single CPUVP, it is possible that the first thread will cause the entire buffer pool to be exausted and the pages overlaid. OK --- then the second thread finally gets control of the CPUVP. Well --- all of the pages have to be reread because we overlaid the original ones read. So by adding a CPUVP in this 'for instance' case, we are possibably able to get to the second thread to process those pages before they get overlaid. Humm --- by the second thread touching the pages, we esclate it's sticking power within the buffer, so that it stays in the buffer even longer, so it might still be arround when the first thread finishes, or gives up control and a third one comes along. Of course this is a theoretical 'for instance' and requires voodoo to get all of the pieces working just right to make it happen. Really a highly rare occurance. Has to be the first full moon after the summer equinox and the wolves are howling, etc... ;-) But yes --- a theoretical possibility. I'd check to see what the IO difference between the 1CPUVP and the 2CPUVP test is. "Neil Truby" <neil.truby@ardenta.com> wrote in message news:37ki8vF5etas8U1@individual.net... > It was 9.21 UC3 btw, not 9.20. > > Are you saying that you can give a plausible explanation of why it will > never work on a 1-cpu box? > > "Madison Pruet" <mpruet@comcast.net> wrote in message > news:3MSdnYl8fIYXYYnfRVn-gg@comcast.com... > > ARGGGGGGGGHHH!!!!! > > > > IDS uses non-preemptive scheduling. The OS uses preemptive (i.e. > > timeslice)... > > > > M.P. > > "Madison Pruet" <mpruet@comcast.net> wrote in message > > news:KcOdnTEoYtadYYnfRVn-vw@comcast.com... > >> IDS uses preemptive scheduling, the OS uses preemptive with timeslice. > >> Sometimes it's best to let the CPUVPS 'duke it out' by over configuring a > >> bit so you get a bit of timeslice scheduling. ;-) > >> > >> My guess is that there are other things causing the hit. For instance, > >> if > > a > >> thread is a bit of a hog, then it could end up doing things like flushing > >> the buffer pool because you're running with 1 cpuvp. In that case if you > >> have two transactions running that are both hogs, then they both flush > >> the > >> buffer pool, but do it serially. So you would be able to get better > >> performance by trying to run them in parallel. At least that way you > > might > >> still have a page in the buffer pool that can be used by the other > > there --- > >> sorta buddy bunch... > >> > >> > >> M.P. > >> > >> > >> "Neil Truby" <neil.truby@ardenta.com> wrote in message > >> news:37k8t6F5fm45bU1@individual.net... > >> > 9.20 UC3 on Red Hat 7. > >> > > >> > This site is running the time series datablade in production on a 2-cpu > >> box. > >> > It's all working fine. > >> > > >> > For reasons we needn't go into they want to move to a single cpu box. > >> > They've set up an identical test server. When we run the same programs > >> with > >> > the live setting (NUMCPUVPS=2) we get the same consistent results. > >> However, > >> > if we reduce NUMCPUVPS to 1 then (and regardless of whether there are 1 > > or > >> 2 > >> > physical cpus present in the test box) the tests run 15-20 times more > >> > slowly. The reason isn't difficult to see: onstat -g rea shows the > > ready > >> > queue with 3 or more entries, indicating a bottleneck on cpu vps. > >> > > >> > The one test I didn't try, through lack of time, is running with > >> > MULTIPROCESSOR=1, NUMCPUVPS=2 with a single physical cpu. > >> > > >> > Anyway, I infer from all this that they need to run this version of IDS > >> and > >> > the TimeSeries datablade on a 2-way box. But the customer isn't happy > >> with > >> > this, and wants a decent and plausible explanation as to why this > >> > cannot > >> be > >> > made to work on a single cpu server. I don't know enought about the > >> > datablade to give a more detailed opinion. > >> > > >> > Any comment gratefully received, as ever. > >> > > >> > thanks > >> > Neil > >> > > >> > > >> > >> > > > > > >