The ready queue Full but Unix not busy

We are experiencing an engine go-slows with all queries take much longer than they should. The ready queue (onstat -g rea) constantly shows 10+ threads, but Unix is only 30% busy (no waiting on IO) ? Can anyone explain the lngspins column ? Geoffrey.poole@ogs.co.uk <mailto:Geoffrey.poole@ogs.co.uk> onstat -g glo INFORMIX-Universal Server Version 9.14.UC2 -- On-Line -- Up 8 days 10:00:14 -- 279184 Kbytes MT global info: sessions threads vps lngspins 6 37 14 446935 sched calls thread switches yield 0 yield n yield forever total: 49010147 12157935 39869397 136564 4116090 per sec: 12 12 0 0 6 Virtual processor summary: class vps usercpu syscpu total cpu 3 52163.56 462.55 52626.11 aio 4 5.58 10.66 16.24 tli 1 60.98 150.49 211.47 lio 2 0.86 3.65 4.51 pio 2 0.74 3.22 3.96 adm 1 0.63 2.35 2.98 msc 1 5.69 5.81 11.50 total 14 52238.04 638.73 52876.77 Individual virtual processors: vp pid class usercpu syscpu total 1 11578 cpu 15158.83 166.28 15325.11 2 11579 adm 0.63 2.35 2.98 3 11580 cpu 22473.47 129.01 22602.48 4 11581 cpu 14531.26 167.26 14698.52 5 11582 lio 0.50 2.15 2.65 6 11583 pio 0.44 1.89 2.33 7 11584 aio 3.00 3.92 6.92 8 11585 msc 5.69 5.81 11.50 9 11586 aio 1.64 3.48 5.12 10 11587 aio 0.55 1.79 2.34 11 11588 aio 0.39 1.47 1.86 12 11589 tli 60.98 150.49 211.47 13 11620 lio 0.36 1.50 1.86 14 11621 pio 0.30 1.33 1.63 tot 52238.04 638.73 52876.77

Can anyone explain the lngspins column ? > MT global info: > sessions threads vps lngspins > 6 37 14 446935 > Spin is situation, when vp wants to acquire latch on the resource, but has to wait, because resource is locked. Long spin - when VP spins more than 10000 times before acquire latch on resource. Probably, you have bad written software .. :))) Regards, Yuri.

Generally looking at onstat -g ath will give a better idea when this type of problem happens. Often it's more an issue of a single thread acting up. Are you using any UDRs ? It might be that you are using one that is 'ill behaved' and is blocking on somthing besides IO. You might also want to look at 'truss' or 'ps' when this occurs. Geoffrey Poole wrote: > We are experiencing an engine go-slows with all queries take much longer > than they should. The ready queue (onstat -g rea) constantly shows 10+ > threads, but Unix is only 30% busy (no waiting on IO) ? Can anyone explain > the lngspins column ? > Geoffrey.poole@ogs.co.uk <mailto:Geoffrey.poole@ogs.co.uk> > > onstat -g glo > INFORMIX-Universal Server Version 9.14.UC2 -- On-Line -- Up 8 days > 10:00:14 -- 279184 Kbytes> > MT global info: > sessions threads vps lngspins > 6 37 14 446935 > > sched calls thread switches yield 0 yield n yield forever > total: 49010147 12157935 39869397 136564 4116090 > per sec: 12 12 0 0 6 > > Virtual processor summary: > class vps usercpu syscpu total > cpu 3 52163.56 462.55 52626.11 > aio 4 5.58 10.66 16.24 > tli 1 60.98 150.49 211.47 > lio 2 0.86 3.65 4.51 > pio 2 0.74 3.22 3.96 > adm 1 0.63 2.35 2.98 > msc 1 5.69 5.81 11.50 > total 14 52238.04 638.73 52876.77 > > Individual virtual processors: > vp pid class usercpu syscpu total > 1 11578 cpu 15158.83 166.28 15325.11 > 2 11579 adm 0.63 2.35 2.98 > 3 11580 cpu 22473.47 129.01 22602.48 > 4 11581 cpu 14531.26 167.26 14698.52 > 5 11582 lio 0.50 2.15 2.65 > 6 11583 pio 0.44 1.89 2.33 > 7 11584 aio 3.00 3.92 6.92 > 8 11585 msc 5.69 5.81 11.50 > 9 11586 aio 1.64 3.48 5.12 > 10 11587 aio 0.55 1.79 2.34 > 11 11588 aio 0.39 1.47 1.86 > 12 11589 tli 60.98 150.49 211.47 > 13 11620 lio 0.36 1.50 1.86 > 14 11621 pio 0.30 1.33 1.63 > tot 52238.04 638.73 52876.77

Just a thought! I notice that you have 3 CPU VPs running. If the server has 8-10 cpus, you would get ~ 30% utilization at the server level under the conditions you have mentioned. You would also find that, if you drill down to the individual cpu utilizations, 3 of them would be at 100%. If this is your case, the solution is really simple - just add CPU vps (this can be done online - "onmode -p +1 cpu" to add a cpu VP). Rudy Fernandes In article <7fkocm$q46$1@news.xmission.com>, Geoffrey Poole <Geoffrey.Poole@ogs.co.uk> wrote: > > We are experiencing an engine go-slows with all queries take much longer > than they should. The ready queue (onstat -g rea) constantly shows 10+ > threads, but Unix is only 30% busy (no waiting on IO) ? Can anyone explain > the lngspins column ? > Geoffrey.poole@ogs.co.uk <mailto:Geoffrey.poole@ogs.co.uk> > > onstat -g glo > INFORMIX-Universal Server Version 9.14.UC2 -- On-Line -- Up 8 days > 10:00:14 -- 279184 Kbytes> > MT global info: > sessions threads vps lngspins > 6 37 14 446935 > > sched calls thread switches yield 0 yield n yield forever > total: 49010147 12157935 39869397 136564 4116090 > per sec: 12 12 0 0 6 > > Virtual processor summary: > class vps usercpu syscpu total > cpu 3 52163.56 462.55 52626.11 > aio 4 5.58 10.66 16.24 > tli 1 60.98 150.49 211.47 > lio 2 0.86 3.65 4.51 > pio 2 0.74 3.22 3.96 > adm 1 0.63 2.35 2.98 > msc 1 5.69 5.81 11.50 > total 14 52238.04 638.73 52876.77 > > Individual virtual processors: > vp pid class usercpu syscpu total > 1 11578 cpu 15158.83 166.28 15325.11 > 2 11579 adm 0.63 2.35 2.98 > 3 11580 cpu 22473.47 129.01 22602.48 > 4 11581 cpu 14531.26 167.26 14698.52 > 5 11582 lio 0.50 2.15 2.65 > 6 11583 pio 0.44 1.89 2.33 > 7 11584 aio 3.00 3.92 6.92 > 8 11585 msc 5.69 5.81 11.50 > 9 11586 aio 1.64 3.48 5.12 > 10 11587 aio 0.55 1.79 2.34 > 11 11588 aio 0.39 1.47 1.86 > 12 11589 tli 60.98 150.49 211.47 > 13 11620 lio 0.36 1.50 1.86 > 14 11621 pio 0.30 1.33 1.63 > tot 52238.04 638.73 52876.77 > > -----------== Posted via Deja News, The Discussion Network ==---------- http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own