Sporadic performance issue

IDS 10.0UC4W3 on RedHat 4 AS The customer's database server, which has been running happily for months, has started, sporadically, to experience very high loads as measured by top on Saturdays (the busiest time of the week). At these times the Informix ready queue is sky-high - perhaps as many as 40 entries. We also notice that the temp dbspaces are getting really hammered. Then, for no apparent reason, it eases off. Co-incidentally or not, we've started to see messages like this in the past few weeks: 23:06:03 Contiguous shared memory segment allocation failed at 0xb73ff000.Allocation successful at 0xffffffff. Check SHMBASE is consistent with the value in $INFORMIXDIR/etc/onconfig.std. If you are using the correct SHMBASE value in your ONCONFIG file, then consider this message informational only. Recycling Informix, or even the server, doesn't fix it. It's evidently not a known problem as we've had a call open with IBM for over a week (PMR 94346,999,866 refers), and they've found nothing matching in the faults fatabase. I wonder if there is a memory fault. Yesterday I tried to re-start the server with 819200 SHMVIRTSIZE and 625000 2k buffers (up from 768000 and 500000) and got a "can't create virtual segment" message. This is a 10g server dedicated to this one instance so I think it should be capable of doing so. Could a memory problem cause a sporadic but dramatic increase in the need to use temo dbspaces for sorting or something? All comments gratefully received! Thanks Neil

Neil Truby wrote: > IDS 10.0UC4W3 on RedHat 4 AS > > The customer's database server, which has been running happily for months, > has started, sporadically, to experience very high loads as measured by top > on Saturdays (the busiest time of the week). > > At these times the Informix ready queue is sky-high - perhaps as many as 40 > entries. We also notice that the temp dbspaces are getting really hammered. > Then, for no apparent reason, it eases off. > > Co-incidentally or not, we've started to see messages like this in the past > few weeks: > > 23:06:03 Contiguous shared memory segment allocation failed at 0xb73ff000.> Allocation successful at 0xffffffff. > Check SHMBASE is consistent with the value in $INFORMIXDIR/etc/onconfig.std. > If you are using the correct SHMBASE value in your ONCONFIG file, then > consider this message informational only. > > Recycling Informix, or even the server, doesn't fix it. It's evidently not > a known problem as we've had a call open with IBM for over a week (PMR > 94346,999,866 refers), and they've found nothing matching in the faults > fatabase. > > I wonder if there is a memory fault. Yesterday I tried to re-start the > server with 819200 SHMVIRTSIZE and 625000 2k buffers (up from 768000 and > 500000) and got a "can't create virtual segment" message. This is a 10g > server dedicated to this one instance so I think it should be capable of > doing so. > > Could a memory problem cause a sporadic but dramatic increase in the need to > use temo dbspaces for sorting or something? > > All comments gratefully received! > > Thanks > Neil > > What is SHMBASE set to? Have you done a search on the IBM thing? Try entering SHMBASE and LINUX into this and hit return http://www-306.ibm.com/software/data/informix/ids/support/ What does onstat -g seg report when you hit the "limit" ... 32 bit absolute max (if you really squeeze things in) is 2.75 Gig of shared memory?

"TBP" <TheBigPotato@NotHere.Co.Uk> wrote in message news:scwRg.32880$Mh2.490@newsfe6-win.ntli.net... >> What is SHMBASE set to? [root~] onstat -c | grep SHMBASE SHMBASE 0x44000000L # Shared memory base address >> What does onstat -g seg report when you hit the "limit" ... 32 bit >> absolute max (if you really squeeze things in) is 2.75 Gig of shared >> memory? [root~] onstat -g seg IBM Informix Dynamic Server Version 10.00.UC4W3 -- Read-Only (Sec) -- Up 1 days 02:02:02 -- 1871912 Kbytes Segment Summary: id key addr size ovhd class blkused blkfree 131596288 1381451777 44000000 1129865216 251368 R* 275843 3 132710435 1381451811 87586000 786432000 24648 V 7163 184837 133496892 1381451835 b6386000 540672 672 M 132 0 Total: - - 1916837888 - - 283138 184840 (* segment locked in memory)

"Neil Truby" <neil.truby@ardenta.com> wrote in message news:4nn46tFb46baU1@individual.net... > IDS 10.0UC4W3 on RedHat 4 AS > > The customer's database server, which has been running happily for months, > has started, sporadically, to experience very high loads as measured by > top on Saturdays (the busiest time of the week). ... Also, although this box is 64-bit compliant and has a meaty 10g of memory, it was only installed with a 32-bit version of RedHat, and therefore IDS. Think this would make much difference?

Neil Truby wrote: > "TBP" <TheBigPotato@NotHere.Co.Uk> wrote in message > news:scwRg.32880$Mh2.490@newsfe6-win.ntli.net... > >>> What is SHMBASE set to? > > [root~] onstat -c | grep SHMBASE > SHMBASE 0x44000000L # Shared memory base address> >>> What does onstat -g seg report when you hit the "limit" ... 32 bit >>> absolute max (if you really squeeze things in) is 2.75 Gig of shared >>> memory? > > [root~] onstat -g seg > > IBM Informix Dynamic Server Version 10.00.UC4W3 -- Read-Only (Sec) -- Up 1 > days 02:02:02 -- 1871912 Kbytes> > Segment Summary: > id key addr size ovhd class blkused blkfree > 131596288 1381451777 44000000 1129865216 251368 R* 275843 3 > 132710435 1381451811 87586000 786432000 24648 V 7163 184837 > 133496892 1381451835 b6386000 540672 672 M 132 0 > Total: - - 1916837888 - - 283138 184840 > > (* segment locked in memory) > > Allright, how about searching on this document : How to allocate a large memory space for Informix shared memory segments on Red Hat Linux 3 (RHEL3) Which will show you how to allocate up to 2.75 Gig. Of course, if you want informix to run out of memory at about 1.87 Gig and then start spilling sorts / hash overflows out to temporary dbspaces, then don't.

"TBP" <TheBigPotato@NotHere.Co.Uk> wrote in message news:sSCRg.32928$Mh2.29482@newsfe6-win.ntli.net... > Neil Truby wrote: >> "TBP" <TheBigPotato@NotHere.Co.Uk> wrote in message >> news:scwRg.32880$Mh2.490@newsfe6-win.ntli.net... >> > Allright, how about searching on this document : > > How to allocate a large memory space for Informix shared memory segments > on Red Hat Linux 3 (RHEL3) > > Which will show you how to allocate up to 2.75 Gig. > > Of course, if you want informix to run out of memory at about 1.87 Gig and > then start spilling sorts / hash overflows out to temporary dbspaces, then > don't. Looks interesting. So, I could do this, or I could just reduce the BUFFERS setting? Would the inability to allocate contiguoyus memory cause thrashing on the temp dbspaces then? Is this just a 32-bit thing btw? If I installed 64-bit Linux and IDS would this go away? Thx Neil

"TBP" <TheBigPotato@NotHere.Co.Uk> wrote in message news:sSCRg.32928$Mh2.29482@newsfe6-win.ntli.net... > > Allright, how about searching on this document : > > How to allocate a large memory space for Informix shared memory segments > on Red Hat Linux 3 (RHEL3) > > Which will show you how to allocate up to 2.75 Gig. > > Of course, if you want informix to run out of memory at about 1.87 Gig and > then start spilling sorts / hash overflows out to temporary dbspaces, then > don't. Where did you get 1.87g from ...? In fact, the IBM doc relates to increasing SHMVIRTSIZE beyond 768m, and refers to a SHMBASE 0x40000000L, whereas it's now recommended to be 0x44 .... The problem appears to be the TOTAL amount of memory that can be allocated. This is the situation at the limit: any increase in SHMVIRTSIZE or BUFFERS and IDS won't start: Segment Summary: id key addr size ovhd class blkused blkfree 181338173 1381451836 43f7c000 540672 672 M 132 0 179404800 1381451777 44000000 1022361600 248080 R* 249597 3 180420640 1381451808 80f00000 921600000 28776 V 3213 221787 Total: - - 1944502272 - - 252942 221790 But, providing I reduce BUFFERS accordingly, I can start IDS with a 1,125,000 SHMVIRTZISE: 192413696 1381451777 44000000 21639168 217544 R* 5280 3 192446465 1381451778 454a3000 1152000000 35808 V 3113 278137 193593381 1381451813 89f45000 540672 672 M 132 0 Total: - - 1174179840 - - 8525 278140