HP Shared Memory Limits

I'm running Informix 7.30.UC7XB on a dedicated 4-processor HP T520 (OS level HP 10.20) with 1.75 GB physical memory and 5.5 GB swap space. Informix will fail to start with a ENOMEM error if we try to allocate more than approximately 1.3G of memory (R, V, and M combined), although no single portion is larger than 1G. It doesn't seem to matter how we mix and match the allocation (through SHMVIRTSIZE and BUFFERS), if we exceed that number we can't start. We can, however, start with that number and add a 200M virtual segment without problems (other than performance) once Informix is up and running, using onmode. Currently we start with SHMVIRTSIZE of 505000 and BUFFERS of 345000 and LOCKS of 1350000. What we can't figure out is where the limitation on initial memory coming from? We are wanting to increase buffers even more and are willing to add physical memory to the system so that we can force residency on all segments, but if we can't get past the 1.3 GB startup, there is not much hope of increasing buffers significantly, since we don't want to lower SHMVIRTSIZE any further (and actually want to return SHMVIRTSIZE to our previous setting of 800000 while retaining/increasing the BUFFER setting of 345000 ). Decreasing SHMVIRTSIZE will require us to continue to add a virtual segment, which has a detrimental effect on performance. We believe that the limit is not based on physical memory size, since we cannot exceed the 1.3G on a 2G machine either (with triple memory for swap space also). Any ideas or suggestions will be gratefully accepted. (This is one more in our series of continuing performance 'opportunities' with 7.30.UCXB and SAP) Thanks, Doug Agnew dagnew@charlottepipe.com

Check Informix release notes and HPUX parameters for IPC. -- --------------------------------------------------------- Steven Hauser email: hause011@tc.umn.edu URL: http://www.tc.umn.edu/~hause011 ---------------------------------------------------------

When using HP-UX 10.20, with the appropriate patches in place, the system-wide shared memory maximum can be extended from 1.75GB to 2.75GB. 1. PHSS_10926: s700_800 10.01-X0 ld(1) and developers tools cumulative PHKL_11086: s800 10.20 LVM, exec, ptrace, MMF, shmem, buf cache, pstat, pdir PHCO_10964: s700_800 10.20 LVM commands cumulative patch PHKL_8327: advises 'chatr'ing the executables PHSS_15380: enables HP's debugger to attach to processes that have been set to SHMEM_MAGIC. 2. As root run 'chatr +pd 64M +pi 4M $INFORMIXDIR/bin/oninit' 3. ... and shmmax in kernel ;-) Eugene In article <ezJg3.4110$hf1.714@news4.mco>, "Doug Agnew" <dagnew@charlottepipe.com> wrote: > I'm running Informix 7.30.UC7XB on a dedicated 4-processor HP T520 (OS level > HP 10.20) with 1.75 GB physical memory and 5.5 GB swap space. > > Informix will fail to start with a ENOMEM error if we try to allocate more > than approximately 1.3G of memory (R, V, and M combined), although no single > portion is larger than 1G. It doesn't seem to matter how we mix and match > the allocation (through SHMVIRTSIZE and BUFFERS), if we exceed that number > we can't start. We can, however, start with that number and add a 200M > virtual segment without problems (other than performance) once Informix is > up and running, using onmode. Currently we start with SHMVIRTSIZE of 505000 > and BUFFERS of 345000 and LOCKS of 1350000. > > What we can't figure out is where the limitation on initial memory coming > from? We are wanting to increase buffers even more and are willing to add > physical memory to the system so that we can force residency on all > segments, but if we can't get past the 1.3 GB startup, there is not much > hope of increasing buffers significantly, since we don't want to lower > SHMVIRTSIZE any further (and actually want to return SHMVIRTSIZE to our > previous setting of 800000 while retaining/increasing the BUFFER setting of > 345000 ). Decreasing SHMVIRTSIZE will require us to continue to add a > virtual segment, which has a detrimental effect on performance. > > We believe that the limit is not based on physical memory size, since we > cannot exceed the 1.3G on a 2G machine either (with triple memory for swap > space also). > > Any ideas or suggestions will be gratefully accepted. > > (This is one more in our series of continuing performance 'opportunities' > with 7.30.UCXB and SAP) > > Thanks, > > Doug Agnew > dagnew@charlottepipe.com > > Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.

We're trying to set up shm-magic on a test machine. BUT THAT IS NOT THE ISSUE..... The limit we're bumping up against and don't understand is a 1.3Gb limit, NOT 1.75G -- and we don't see anything else in our system taking up .45G shared memory. After all, once we bring up the 1.3Gb Informix server, we can then add 200M or more via onmode. Our question is 'what could be causing the 1.3Gb barrier'. SHMMAX (at 1G) and all other parameters are currently at SAP defined values. Doug Eugene Nechayev <4new@mindless.com> wrote in message <7m08o9$lhn$1@nnrp1.deja.com>... >When using HP-UX 10.20, with the appropriate patches in place, the >system-wide shared memory maximum can be extended from 1.75GB to >2.75GB. > >1. >PHSS_10926: s700_800 10.01-X0 ld(1) and developers tools cumulative >PHKL_11086: s800 10.20 LVM, exec, ptrace, MMF, shmem, buf cache, >pstat, pdir >PHCO_10964: s700_800 10.20 LVM commands cumulative patch >PHKL_8327: advises 'chatr'ing the executables >PHSS_15380: enables HP's debugger to attach to processes that have >been set to SHMEM_MAGIC. > >2. As root run 'chatr +pd 64M +pi 4M $INFORMIXDIR/bin/oninit' > >3. ... and shmmax in kernel ;-) > >Eugene > > >In article <ezJg3.4110$hf1.714@news4.mco>, > "Doug Agnew" <dagnew@charlottepipe.com> wrote: >> I'm running Informix 7.30.UC7XB on a dedicated 4-processor HP T520 (OS level >> HP 10.20) with 1.75 GB physical memory and 5.5 GB swap space. >> >> Informix will fail to start with a ENOMEM error if we try to allocate more >> than approximately 1.3G of memory (R, V, and M combined), although no single >> portion is larger than 1G. It doesn't seem to matter how we mix and match >> the allocation (through SHMVIRTSIZE and BUFFERS), if we exceed that number >> we can't start. We can, however, start with that number and add a 200M >> virtual segment without problems (other than performance) once Informix is >> up and running, using onmode. Currently we start with SHMVIRTSIZE of 505000 >> and BUFFERS of 345000 and LOCKS of 1350000. >> >> What we can't figure out is where the limitation on initial memory coming >> from? We are wanting to increase buffers even more and are willing to add >> physical memory to the system so that we can force residency on all >> segments, but if we can't get past the 1.3 GB startup, there is not much >> hope of increasing buffers significantly, since we don't want to lower >> SHMVIRTSIZE any further (and actually want to return SHMVIRTSIZE to our >> previous setting of 800000 while retaining/increasing the BUFFER setting of >> 345000 ). Decreasing SHMVIRTSIZE will require us to continue to add a >> virtual segment, which has a detrimental effect on performance. >> >> We believe that the limit is not based on physical memory size, since we >> cannot exceed the 1.3G on a 2G machine either (with triple memory for swap >> space also). >> >> Any ideas or suggestions will be gratefully accepted. >> >> (This is one more in our series of continuing performance 'opportunities' >> with 7.30.UCXB and SAP) >> >> Thanks, >> >> Doug Agnew >> dagnew@charlottepipe.com >> >> > > >Sent via Deja.com http://www.deja.com/ >Share what you know. Learn what you don't.

In article <2hOg3.4567$hf1.741@news4.mco>, "Doug Agnew" <dagnew@charlottepipe.com> wrote: > We're trying to set up shm-magic on a test machine. BUT THAT IS NOT THE > ISSUE..... The limit we're bumping up against and don't understand is a > 1.3Gb limit, NOT 1.75G -- and we don't see anything else in our system > taking up .45G shared memory. After all, once we bring up the 1.3Gb > Informix server, we can then add 200M or more via onmode. > > Our question is 'what could be causing the 1.3Gb barrier'. > > SHMMAX (at 1G) and all other parameters are currently at SAP defined values. > Okay. The process address space is divided into four quadrants. On a 32 GB system, these quadrants are each 1 GB. The size of the quadrants limits the amount of address space available to processes. Each quadrant is used to store different types of information for the process. On 10.X 32 bit, the default usage for the quadrants is: Quadrant 1 Process text Quadrant 2 Process data Quadrant 3 Shared libraries Shared memory Memory mapped files Quadrant 4 Shared libraries Shared memory Memory mapped files The last part of quadrant 4 is reserved for system I/O (0.25GB). After starting you have 1.75 GB - HP-UX shared libraries.=? You can check it "ipcs -b". Also you'll see REAL informix shared memory segments size that shows the command. Usually the size is bigger than size in "onstat -g seg" A single shared memory segment cannot cross quadrant boundaries. You have two Informix segment in shared memory after starting, but if one segment crossed quadrant boundaries you would have a Informix error. shmem_magic should help you. Regards, Eugene > Eugene Nechayev <4new@mindless.com> wrote in message > <7m08o9$lhn$1@nnrp1.deja.com>... > >When using HP-UX 10.20, with the appropriate patches in place, the > >system-wide shared memory maximum can be extended from 1.75GB to > >2.75GB. > > > >1. > >PHSS_10926: s700_800 10.01-X0 ld(1) and developers tools cumulative > >PHKL_11086: s800 10.20 LVM, exec, ptrace, MMF, shmem, buf cache, > >pstat, pdir > >PHCO_10964: s700_800 10.20 LVM commands cumulative patch > >PHKL_8327: advises 'chatr'ing the executables > >PHSS_15380: enables HP's debugger to attach to processes that have > >been set to SHMEM_MAGIC. > > > >2. As root run 'chatr +pd 64M +pi 4M $INFORMIXDIR/bin/oninit' > > > >3. ... and shmmax in kernel ;-) > > > >Eugene > > > > > >In article <ezJg3.4110$hf1.714@news4.mco>, > > "Doug Agnew" <dagnew@charlottepipe.com> wrote: > >> I'm running Informix 7.30.UC7XB on a dedicated 4-processor HP T520 (OS > level > >> HP 10.20) with 1.75 GB physical memory and 5.5 GB swap space. > >> > >> Informix will fail to start with a ENOMEM error if we try to allocate > more > >> than approximately 1.3G of memory (R, V, and M combined), although no > single > >> portion is larger than 1G. It doesn't seem to matter how we mix and > match > >> the allocation (through SHMVIRTSIZE and BUFFERS), if we exceed that > number > >> we can't start. We can, however, start with that number and add a 200M > >> virtual segment without problems (other than performance) once Informix > is > >> up and running, using onmode. Currently we start with SHMVIRTSIZE of > 505000 > >> and BUFFERS of 345000 and LOCKS of 1350000. > >> > >> What we can't figure out is where the limitation on initial memory coming > >> from? We are wanting to increase buffers even more and are willing to > add > >> physical memory to the system so that we can force residency on all > >> segments, but if we can't get past the 1.3 GB startup, there is not much > >> hope of increasing buffers significantly, since we don't want to lower > >> SHMVIRTSIZE any further (and actually want to return SHMVIRTSIZE to our > >> previous setting of 800000 while retaining/increasing the BUFFER setting > of > >> 345000 ). Decreasing SHMVIRTSIZE will require us to continue to add a > >> virtual segment, which has a detrimental effect on performance. > >> > >> We believe that the limit is not based on physical memory size, since we > >> cannot exceed the 1.3G on a 2G machine either (with triple memory for > swap > >> space also). > >> > >> Any ideas or suggestions will be gratefully accepted. > >> > >> (This is one more in our series of continuing performance 'opportunities' > >> with 7.30.UCXB and SAP) > >> > >> Thanks, > >> > >> Doug Agnew > >> dagnew@charlottepipe.com > >> > >> > > > > > >Sent via Deja.com http://www.deja.com/ > >Share what you know. Learn what you don't. > > Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.

Well folks, thanks for all the advice. We finally got over the 1.3G limit -- don't know what causes it, but we got around it. Turns out that IF you allocate enough buffers and locks to put the resident segment at 1G (and hence fill quad 3), then the virtual segment can be larger. Somehow, having two small segments caused some strange behaviour that locked in at 1.3G. So, now, we're pushing 1.6 and as soon as we get more memory (1.75G on machine now), we'll max out both resident and virtual shared memory segments. Thanks, Doug Agnew In article <7m0gc6$p1d$1@nnrp1.deja.com>, Eugene Nechayev <4new@mindless.com> wrote: > In article <2hOg3.4567$hf1.741@news4.mco>, > "Doug Agnew" <dagnew@charlottepipe.com> wrote: > > We're trying to set up shm-magic on a test machine. BUT THAT IS NOT THE > > ISSUE..... The limit we're bumping up against and don't understand is a > > 1.3Gb limit, NOT 1.75G -- and we don't see anything else in our system > > taking up .45G shared memory. After all, once we bring up the 1.3Gb > > Informix server, we can then add 200M or more via onmode. > > > > Our question is 'what could be causing the 1.3Gb barrier'. > > > > SHMMAX (at 1G) and all other parameters are currently at SAP defined values. > > > Okay. > > The process address space is divided into four quadrants. On a 32 GB system, > these quadrants are each 1 GB. The size of the quadrants limits the amount > of address space available to processes. Each quadrant is used to store > different types of information for the process. On 10.X 32 bit, the > default usage for the quadrants is: > > Quadrant 1 Process text > Quadrant 2 Process data > Quadrant 3 Shared libraries > Shared memory > Memory mapped files > Quadrant 4 Shared libraries > Shared memory > Memory mapped files > > The last part of quadrant 4 is reserved for system I/O (0.25GB). > > After starting you have 1.75 GB - HP-UX shared libraries.=? You can check it > "ipcs -b". Also you'll see REAL informix shared memory segments size that > shows the command. Usually the size is bigger than size in "onstat -g seg" > > A single shared memory segment cannot cross quadrant boundaries. You have two > Informix segment in shared memory after starting, but if one segment crossed > quadrant boundaries you would have a Informix error. > > shmem_magic should help you. > > Regards, > > Eugene > > > Eugene Nechayev <4new@mindless.com> wrote in message > > <7m08o9$lhn$1@nnrp1.deja.com>... > > >When using HP-UX 10.20, with the appropriate patches in place, the > > >system-wide shared memory maximum can be extended from 1.75GB to > > >2.75GB. > > > > > >1. > > >PHSS_10926: s700_800 10.01-X0 ld(1) and developers tools cumulative > > >PHKL_11086: s800 10.20 LVM, exec, ptrace, MMF, shmem, buf cache, > > >pstat, pdir > > >PHCO_10964: s700_800 10.20 LVM commands cumulative patch > > >PHKL_8327: advises 'chatr'ing the executables > > >PHSS_15380: enables HP's debugger to attach to processes that have > > >been set to SHMEM_MAGIC. > > > > > >2. As root run 'chatr +pd 64M +pi 4M $INFORMIXDIR/bin/oninit' > > > > > >3. ... and shmmax in kernel ;-) > > > > > >Eugene > > > > > > > > >In article <ezJg3.4110$hf1.714@news4.mco>, > > > "Doug Agnew" <dagnew@charlottepipe.com> wrote: > > >> I'm running Informix 7.30.UC7XB on a dedicated 4-processor HP T520 (OS > > level > > >> HP 10.20) with 1.75 GB physical memory and 5.5 GB swap space. > > >> > > >> Informix will fail to start with a ENOMEM error if we try to allocate > > more > > >> than approximately 1.3G of memory (R, V, and M combined), although no > > single > > >> portion is larger than 1G. It doesn't seem to matter how we mix and > > match > > >> the allocation (through SHMVIRTSIZE and BUFFERS), if we exceed that > > number > > >> we can't start. We can, however, start with that number and add a 200M > > >> virtual segment without problems (other than performance) once Informix > > is > > >> up and running, using onmode. Currently we start with SHMVIRTSIZE of> > 505000 > > >> and BUFFERS of 345000 and LOCKS of 1350000. > > >> > > >> What we can't figure out is where the limitation on initial memory coming > > >> from? We are wanting to increase buffers even more and are willing to > > add > > >> physical memory to the system so that we can force residency on all > > >> segments, but if we can't get past the 1.3 GB startup, there is not much > > >> hope of increasing buffers significantly, since we don't want to lower > > >> SHMVIRTSIZE any further (and actually want to return SHMVIRTSIZE to our > > >> previous setting of 800000 while retaining/increasing the BUFFER setting > > of > > >> 345000 ). Decreasing SHMVIRTSIZE will require us to continue to add a > > >> virtual segment, which has a detrimental effect on performance. > > >> > > >> We believe that the limit is not based on physical memory size, since we > > >> cannot exceed the 1.3G on a 2G machine either (with triple memory for > > swap > > >> space also). > > >> > > >> Any ideas or suggestions will be gratefully accepted. > > >> > > >> (This is one more in our series of continuing performance 'opportunities' > > >> with 7.30.UCXB and SAP) > > >> > > >> Thanks, > > >> > > >> Doug Agnew > > >> dagnew@charlottepipe.com > > >> > > >> > > > > > > > > >Sent via Deja.com http://www.deja.com/ > > >Share what you know. Learn what you don't. > > > > > > Sent via Deja.com http://www.deja.com/ > Share what you know. Learn what you don't. > Sent via Deja.com http://www.deja.com/ Share what you know. Learn what you don't.

dagnew@charlottepipe.com wrote: As you discovered, you were getting bit by (the inability to span) quadrant boundaries. If you make the R segment >768 MB, you will force it into the 1GB quadrant, as opposed to the .75GB one. You might just as well set the V segment to .74GB and forget about it. You can do it anytime, you don't have to have enough real memory to back it up, though that's certainly preferable. Greg > > Well folks, thanks for all the advice. We finally got over the 1.3G > limit -- don't know what causes it, but we got around it. > > Turns out that IF you allocate enough buffers and locks to put the > resident segment at 1G (and hence fill quad 3), then the virtual segment > can be larger. Somehow, having two small segments caused some strange > behaviour that locked in at 1.3G. So, now, we're pushing 1.6 and as > soon as we get more memory (1.75G on machine now), we'll max out both > resident and virtual shared memory segments. > > Thanks, > Doug Agnew snip....