Resident memory size vs. Virtual memory size

Hello, Happy New Year ! IDS 11.50 FC8 , Redhat Linux Traditionally, as our experience, Virtual memory size is normally much smaller than the Resident memory size . Now things seems changed. Yes, we have more sessions and more busy works ...., but the Virtual memory consumption seems some thing suspicious .... For example, we assigned 4GB Resident memory and 1GB Virtual memory , it worked well( without big accumulation of too many virtual segments). Now we often have lot of IDS added extra Virtual memory segments ( especially happened more in IDSD11.50 FC8) . It seems to me even the session (which triggered adding virtual segments) is gone, the added virtual segments can not be released by onmode -F . So currently, we are forced to have a big Virtual memory( 4.5 GB) which is bigger than the Resident memory ( 4 GB). Any comments? Thanks, Frank --20cf3071d16a63a14d04d26923dd

Is it possible that your LOCKS table was expanded by a couple of million locks due to a large transactio? That one bit me a few weeks ago. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Thu, Jan 3, 2013 at 3:35 PM, FRANK <yunyaoqu@gmail.com> wrote: > Hello, Happy New Year ! > > IDS 11.50 FC8 , Redhat Linux > > Traditionally, as our experience, Virtual memory size is normally much > smaller than the Resident memory size . > > Now things seems changed. Yes, we have more sessions and more busy works > ....., but the Virtual memory consumption seems some thing suspicious .... > > For example, we assigned 4GB Resident memory and 1GB Virtual memory , > it worked well( without big accumulation of too many virtual segments). > > Now we often have lot of IDS added extra Virtual memory segments ( > especially happened more in IDSD11.50 FC8) . It seems to me even the > session (which triggered adding virtual segments) is gone, the added > virtual segments can not be released by onmode -F . So currently, we are > forced to have a big Virtual memory( 4.5 GB) which is bigger than the > Resident memory ( 4 GB). > > Any comments? > > Thanks, > Frank > > --20cf3071d16a63a14d04d26923dd > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --20cf302239990d02b004d2695807

more sessions = more work = more locks = which means more memory needed for those locks. look in the msg log and you should see "dynamically allocating more locks " (or similiar) if that is the case. But I have to say that 4 gigs is alot so check $ONCONFIG paramater "shmadd" , there may be an extra 0 tacked on the end. As a FYI every time there is a dynamic allocation of memory it is recorded in the messags log so you should be able to quickly decipher the circumstance surrounding the dynamic allocation. Mark

Besides the locks... which is a frequent cause until IBM fixes the issue and implements a limit per session, if you use Java in your applications run: onstat -g ses | sort -nk 7 and look at the last sessions... memory usage column... I'm mentioning Java, but it can happen with any language... It's just more frequent with Java and persistent sessions... sometimes the programmers miss the point of preparing the statement and put the prepare inside the loops (or use methods that implicitly make a prepare). My "personal" record was 1.5GB in a session... But if it gets that far you'll notice a great performance degradation. Anyway, worth the checking.... Regards. On Thu, Jan 3, 2013 at 10:02 PM, MARK JALKIEWICZ < mark.jalkiewicz@verizon.net> wrote: > more sessions = more work = more locks = which means more memory needed for > those locks. > > look in the msg log and you should see "dynamically allocating more locks " > (or similiar) if that is the case. But I have to say that 4 gigs is alot so > check $ONCONFIG paramater "shmadd" , there may be an extra 0 tacked on the > end. As a FYI every time there is a dynamic allocation of memory it is > recorded in the messags log so you should be able to quickly decipher the > circumstance surrounding the dynamic allocation. > > Mark > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --20cf307f333ea813fe04d269bab8

Thanks Fernando, Mark and Art! It is not lock issues here Java session used to exhausted our Virtual memory a lot ( because of its cursor leak, Fernamdo mentioned), but it has been ffixed ..... It might be some little ghosts (created by developers???) are still there ??? Will have a close monitoring ..... Frank On Thu, Jan 3, 2013 at 5:17 PM, Fernando Nunes <domusonline@gmail.com>wrote: > Besides the locks... which is a frequent cause until IBM fixes the issue > and implements a limit per session, if you use Java in your applications > run: > > onstat -g ses | sort -nk 7> > and look at the last sessions... memory usage column... I'm mentioning > Java, but it can happen with any language... It's just more frequent with > Java and persistent sessions... sometimes the programmers miss the point of > preparing the statement and put the prepare inside the loops (or use > methods that implicitly make a prepare). > My "personal" record was 1.5GB in a session... But if it gets that far > you'll notice a great performance degradation. > Anyway, worth the checking.... > > Regards. > > On Thu, Jan 3, 2013 at 10:02 PM, MARK JALKIEWICZ < > mark.jalkiewicz@verizon.net> wrote: > > > more sessions = more work = more locks = which means more memory needed > for > > those locks. > > > > look in the msg log and you should see "dynamically allocating more > locks " > > (or similiar) if that is the case. But I have to say that 4 gigs is alot > so > > check $ONCONFIG paramater "shmadd" , there may be an extra 0 tacked on > the > > end. As a FYI every time there is a dynamic allocation of memory it is > > recorded in the messags log so you should be able to quickly decipher the > > circumstance surrounding the dynamic allocation. > > > > Mark > > > > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently... > > --20cf307f333ea813fe04d269bab8 > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --0016e68deab16cdfbf04d26a256e

We all look back and chuckle at mistakes and discoveries that had us sweating bullets at the time. "My personal record"....... I laughed out loud! I once wiped out an entire development instance by not setting the IFX environment correctly. Bob ----- Original Message ----- From: "Fernando Nunes" <domusonline@gmail.com> To: ids@iiug.org Sent: Thursday, January 3, 2013 5:17:36 PM Subject: Re: Resident memory size vs. Virtual memory size [29180] Besides the locks... which is a frequent cause until IBM fixes the issue and implements a limit per session, if you use Java in your applications run: onstat -g ses | sort -nk 7 and look at the last sessions... memory usage column... I'm mentioning Java, but it can happen with any language... It's just more frequent with Java and persistent sessions... sometimes the programmers miss the point of preparing the statement and put the prepare inside the loops (or use methods that implicitly make a prepare). My "personal" record was 1.5GB in a session... But if it gets that far you'll notice a great performance degradation. Anyway, worth the checking.... Regards. On Thu, Jan 3, 2013 at 10:02 PM, MARK JALKIEWICZ < mark.jalkiewicz@verizon.net> wrote: > more sessions = more work = more locks = which means more memory needed for > those locks. > > look in the msg log and you should see "dynamically allocating more locks " > (or similiar) if that is the case. But I have to say that 4 gigs is alot so > check $ONCONFIG paramater "shmadd" , there may be an extra 0 tacked on the > end. As a FYI every time there is a dynamic allocation of memory it is > recorded in the messags log so you should be able to quickly decipher the > circumstance surrounding the dynamic allocation. > > Mark > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --20cf307f333ea813fe04d269bab8 ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

For the record, "my personal record" means the worst situation I've seen as a DBA... It was not caused by me! :) But it's really "easy" to happen because Java applications tend to use persistent connections, so this kind of situations get more visibility... As for your personal situation, recent versions would probably avoid that... (FULL_DISK_INIT) On Sat, Jan 5, 2013 at 6:36 PM, rroussey@comcast.net <rroussey@comcast.net>wrote: > We all look back and chuckle at mistakes and discoveries that had us > sweating > bullets at the time. > "My personal record"....... I laughed out loud! I once wiped out an entire > development instance by not setting the IFX environment correctly. > > Bob > > ----- Original Message ----- > From: "Fernando Nunes" <domusonline@gmail.com> > To: ids@iiug.org > Sent: Thursday, January 3, 2013 5:17:36 PM > Subject: Re: Resident memory size vs. Virtual memory size [29180] > > Besides the locks... which is a frequent cause until IBM fixes the issue > and implements a limit per session, if you use Java in your applications > run: > > onstat -g ses | sort -nk 7> > and look at the last sessions... memory usage column... I'm mentioning > Java, but it can happen with any language... It's just more frequent with > Java and persistent sessions... sometimes the programmers miss the point of > preparing the statement and put the prepare inside the loops (or use > methods that implicitly make a prepare). > My "personal" record was 1.5GB in a session... But if it gets that far > you'll notice a great performance degradation. > Anyway, worth the checking.... > > Regards. > > On Thu, Jan 3, 2013 at 10:02 PM, MARK JALKIEWICZ < > mark.jalkiewicz@verizon.net> wrote: > > > more sessions = more work = more locks = which means more memory needed > for > > those locks. > > > > look in the msg log and you should see "dynamically allocating more > locks " > > (or similiar) if that is the case. But I have to say that 4 gigs is alot > so > > check $ONCONFIG paramater "shmadd" , there may be an extra 0 tacked on > the > > end. As a FYI every time there is a dynamic allocation of memory it is > > recorded in the messags log so you should be able to quickly decipher the > > circumstance surrounding the dynamic allocation. > > > > Mark > > > > > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently... > > --20cf307f333ea813fe04d269bab8 > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --047d7b66f3c54d5ec904d28fd29d

The following is the output of onstat -g mem, it sounds pool: dictpool has a excessive virtual memory consumption ( around 3GB?), dictpool V 13f5bc040 3083149312 22008 1505143 12 Any idea why " dictpool" is such a big consumer of Virtual segments? Thanks, Frank [informix@maggie ~]$ onstat -g mem IBM Informix Dynamic Server Version 11.50.FC8 -- On-Line -- Up 42 days 19:52:04 -- 8130128 KbytesPool Summary: name class addr totalsize freesize #allocfrag #freefrag CDRGeval8 V 140e89040 147456 26776 92 18 CDRD_40754 V 22ef87040 159744 40584 197 40 CDRD_40763 V 22f947040 159744 43480 183 42 CDRD_40772 V 22fcc4040 135168 34736 178 38 afpool V 13e270040 307200 246576 118 108 tpcpool V 13f5c6040 98304 7912 88 7 grprUpLinks V 140471040 4096 808 1 1 CDRGeval9 V 140eef040 147456 27688 91 18 CDRD_40764 V 22f359040 155648 36664 197 38 CDRD_40773 V 22e434040 159744 43408 183 40 CDRD_40765 V 22f92a040 159744 40568 197 44 seqpool V 13f602040 4096 736 2 1 CDRD_40757 V 22e75b040 159744 40184 203 39 pnlpool V 13f5c9040 90112 3288 82 3 grouper V 14046f040 5791744 2417656 40423 4556 CDRD_40758 V 230e7c040 155648 36272 199 40 sbtlist V 13e617040 20480 7200 4 3 sqcrypto V 13ec84040 4096 464 2 1 dstpool V 13f5c5040 2027520 175528 1723 108 CDRD_40768 V 22ec07040 159744 40760 197 42 CDRGtx23320 V 22c680040 4096 808 1 1 CDRD_40769 V 2285af040 159744 43424 183 39 ampool V 13f5f8040 8192 3464 17 1 shmcon M 21028a040 1085440 4864 2 2 main_loop() V 13f882040 352256 17968 64 22 sb_delundoq V 13e649040 49152 8752 4 3 XTF_mem V 13f659040 716800 6632 4 3 61*O0 V 14313f040 4096 808 1 1 6103734*O0 V 1a0fa2040 4096 808 1 1 22700700 V 223ec1040 155648 52248 246 34 EXE.37.91 V 22d942040 4096 616 4 1 EXE.37.91 V 22bda9040 4096 680 3 1 EXE.37.91 V 22a441040 12288 3584 57 1 37*O0 V 14055d040 4096 808 1 1 22700701 V 223e5a040 159744 57272 236 37 38*O0 V 14328f040 4096 808 1 1 22700702 V 222a82040 155648 52416 246 38 22815110 V 22a6f9040 196608 45184 246 35 EXE.38.92 V 22a378040 4096 744 2 1 EXE.38.92 V 22b937040 4096 680 3 1 EXE.38.92 V 22cb41040 4096 544 5 1 EXE.38.92 V 22ff36040 4096 360 8 1 39*O0 V 142416040 4096 808 1 1 bf_prioswee V 1424b3040 28672 3104 13 3 22700703 V 2242ff040 143360 45544 234 34 CDRGtx23398 V 22df70040 4096 808 1 1 16020616 V 1dd217040 102400 25008 113 20 22700704 V 222476040 139264 42016 229 39 EXE.39.93 V 22ac27040 4096 680 3 1 EXE.39.93 V 227f7c040 4096 744 2 1 EXE.39.93 V 229f58040 4096 544 5 1 EXE.39.93 V 22ecf0040 4096 360 8 1 6103659*O0 V 1a18dc040 4096 808 1 1 22700705 V 222784040 151552 54352 229 42 20580270 V 19fcf8040 98304 21072 113 19 22700670 V 14813a040 159744 55704 256 40 22700706 V 223d60040 139264 41736 234 34 18506302 V 1e5b1f040 12288 416 16 1 22700671 V 211e70040 167936 65008 241 38 22700680 V 14d372040 151552 48016 251 36 22700707 V 22482a040 118784 33152 174 28 22700672 V 212b1e040 151552 37944 259 24 22700681 V 2131b5040 163840 60440 251 36 SES.37.91 V 1405ba040 8192 3736 3 2 3546871*O0 V 1776e7040 4096 808 1 1 16023834 V 1f7da1040 98304 21072 113 19 22523076 V 22828a040 151552 62528 146 38 22664016 V 22974d040 12288 416 16 1 22700682 V 217aea040 155648 52064 251 36 22700718 V 2268e6040 143360 45528 234 33 22731363 V 22bfca040 258048 138360 294 71 22815036 V 22aed5040 196608 51008 247 36 3546881*O0 V 1521f7040 4096 808 1 1 22700683 V 218703040 147456 49872 234 36 22815118 V 22cfa5040 196608 51968 247 35 SES.38.92 V 1425d0040 8192 3736 3 2 4076658*O0 V 182c5b040 4096 808 1 1 4076766*O0 V 17895f040 4096 808 1 1 20580734 V 19d820040 98304 21040 113 18 22700684 V 219f76040 151552 53816 234 40 22849211 V 22e0ce040 196608 51968 247 35 3546874*O0 V 174dba040 4096 808 1 1 3546883*O0 V 184ac8040 4096 808 1 1 3546892*O0 V 179a4a040 4096 808 1 1 22700685 V 21c60c040 143360 45472 234 38 userlbacpoo V 13f5cd040 8192 3288 2 2 SES.39.93 V 142b8f040 8192 3736 3 2 3546875*O0 V 1784dd040 4096 808 1 1 9645923*O0 V 1dc2dc040 4096 808 1 1 22700686 V 21e362040 147456 49320 239 37 22815832 V 22c60c040 102400 33088 124 19 22875106 V 22edf4040 233472 68624 369 54 22887211 V 22feaa040 126976 33608 200 36 22887310 V 194d09040 126976 40320 171 36 3546876*O0 V 191bf0040 4096 808 1 1 3546885*O0 V 179048040 4096 808 1 1 19460453 V 1b1400040 192512 46600 246 36 22700687 V 21f684040 139264 41488 234 33 22887311 V 22dd82040 122880 35840 172 34 22695612 V 22b1f5040 274432 153072 314 73 22764516 V 228c0e040 204800 53120 247 37 22884351 V 230307040 196608 51968 247 35 22887303 V 22fc07040 135168 48416 172 38 22887312 V 22eea2040 126976 39888 172 39 22887510 V 22751d040 200704 81712 247 44 PRP.37.91 V 140556040 4096 408 3 1 16657090 V 1c9f79040 98304 20992 113 19 22843807 V 22c55b040 249856 44192 356 37 22868242 V 22ff7d040 229376 64304 369 57 22880464 V 231c4d040 122880 40112 126 34 22881058 V 2300e1040 114688 34704 122 30 22883182 V 22fe6f040 192512 47952 246 40 22886107 V 22f6f6040 106496 17784 154 20 22887313 V 22b1ae040 122880 35920 172 31 22700699 V 21e5e9040 155648 52472 246 35 22841558 V 22faec040 114688 30912 142 23 22883174 V 231054040 196608 51968 247 35 22885163 V 22fa37040 172032 34888 283 40 22887314 V 22fe90040 122880 35840 172 33 PRP.38.92 V 14039d040 4096 408 3 1 CDRRTBClean V 141c57040 118784 31000 87 17 22815099 V 22d31a040 196608 50368 247 36 22883535 V 22f3a7040 98304 21576 109 17 22884336 V 22b7c8040 196608 50368 247 36 22885038 V 22ee96040 184320 79288 210 48 22885164 V 229212040 241664 111112 303 56 22887315 V 22ad40040 126976 39952 172 36 22887405 V 22e111040 192512 73640 247 43 22887423 V 22e9f9040 167936 54528 214 43 22808764 V 22fd2a040 270336 149848 299 76 22876138 V 22f6dc040 102400 30488 144 22 22885255 V 22f557040 106496 17856 154 20 22887253 V 22d9d4040 126976 33952 200 36 22887316 V 22c2b7040 122880 35824 172 29 22887415 V 2298bd040 135168 47392 177 37 PRP.39.93 V 1403c9040 4096 408 3 1 22866581 V 22c957040 106496 29272 114 21 22883483 V 22e286040 167936 30872 283 39 22885256 V 2289f3040 167936 30968 283 34 22887173 V 23010a040 131072 37896 200 35 22887317 V 22f347040 122880 35792 172 32 22887425 V 22fae2040 143360 48432 213 48 17683365 V 1b2b10040 307200 45880 324 45 22883169 V 22c030040 196608 50368 247 36 22887246 V 22e331040 131072 37928 200 37 22887318 V 22e77d040 126976 40696 170 34 22796950 V 22c2d1040 147456 52032 199 45 22887292 V 22ef95040 167936 54264 214 51 cdrRQM V 14081d040 962560 309464@

Hi, Check for this bug , APAR= IC77372 : ABNORMAL ALLOCATION OF VIRTUAL MEMORY SEGMENTS WHEN CREATE INDEX ON FRAGMENTED TABLES. rgds schillache