DB Instance crash "ipcshm buf_wait semop errno=36"

Hello, We are facing instance crash while executing of following command on HDR Secondary instance, at the time of crash "ipcshm buf_wait semop errno=36" error is displayed on command prompt. oncheck -cD test_db:table-1 after analyzing session detail, i observe the abnormal memory status i.e: total used memory memory 933036032 476864192 And following exception is displayed in af file. Results: Exception Caught. Type: MT_EX_OS, Context: mem Can any one help me for resolving this issue.

VERSION AND PLATFORM INFORMATION?????????????????????????????????????????? Most problems are in some way dependent either on the IDS version or the platform. In this case errno 36 has different meanings on different platforms. On Solaris it is: "Operation already in progress" on Linux it is: "Filename too long". I can guess that this may not be Linux from the context of the IDS message, but... Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) IIUG Board of Directors (art@iiug.org) See you at the 2010 IIUG Informix Conference April 25-28, 2010 Overland Park (Kansas City), KS www.iiug.org/conf Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, Jan 22, 2010 at 6:30 AM, ABRAR RASHID <mabrar@i2cinc.com> wrote: > Hello, > > We are facing instance crash while executing of following command on HDR > Secondary instance, at the time of crash "ipcshm buf_wait semop errno=36" > error is displayed on command prompt. > > oncheck -cD test_db:table-1> > after analyzing session detail, i observe the abnormal memory status i.e: > > total used > memory memory > 933036032 476864192 > > And following exception is displayed in af file. > Results: Exception Caught. Type: MT_EX_OS, Context: mem > > Can any one help me for resolving this issue. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --00163646dbdc299984047dc08e1b

Hello Art, Thanks for your response. Please find below the required details: I hope these would be helpful, however, if you need any further details then do let me know. $ onstat -VIBM Informix Dynamic Server Version 11.50.FC3W1 Software Serial Number AAA#B000000 $ uname -aSunOS DB1 5.10 Generic_141414-08 sun4v sparc SUNW,Sun-Fire-T200 Command Executed on this Instance: oncheck -cD test_db:table-1 Here are the extract from the online.log file ------------------online.log--------------------------------------------- 02:32:21 Logical Log 42461 Complete, timestamp: 0x60dde533. 02:33:01 stack trace for pid 23303 written to /u/informix11.5-3/tmp/af.4807edc 02:33:01 Assert Failed: No Exception Handler 02:33:01 IBM Informix Dynamic Server Version 11.50.FC3W1 02:33:01 Who: Session(55, informix@DB1, 23690, 11b13a668) Thread(152, oncheck, 11b107680, 11) File: mtex.c Line: 491 02:33:01 Results: Exception Caught. Type: MT_EX_OS, Context: mem 02:33:01 Action: Please notify IBM Informix Technical Support. 02:33:01 See Also: /u/informix11.5-3/tmp/af.4807edc, shmem.4807edc.0 Assertion Failure Type: CRASH Host Name: DB1 Database Server Name: ids_sec02 Time of failure: Fri Jan 22 02:33:02 PST 2010 AF file: /u/informix11.5-3/tmp/af.4807edc Shared memory file: None System Blocking: OFF 02:33:57 mtex.c, line 491, thread 152, proc id 23303, No Exception Handler. SENDER IS NULL NO MAIL WILL BE SENT 02:33:57 Fatal error in ADM VP at mt.c:13434 02:33:57 Unexpected virtual processor termination, pid = 23303, exit = 0x100 SENDER IS NULL NO MAIL WILL BE SENT 02:33:58 PANIC: Attempting to bring system down ------------------online.log--------------------------------------------- This is the extract from /u/informix11.5-3/tmp/af.4807edc file ===========------------- - - - - - - /u/informix11.5-3/bin/onstat -g ses 55: IBM Informix Dynamic Server Version 11.50.FC3W1 -- Read-Only (Sec) -- Up 00:18:24 -- 1843200 Kbytes session effective #RSAM total used dynamic id user user tty pid hostname threads memory memory explain 55 informix - 16 23690 DB1 1 933068800 476880976 off tid name rstcb flags curstk status 152 oncheck 11b107680 ---PRR- 465394207 running- Memory pools count 1 name class addr totalsize freesize #allocfrag #freefrag 55 V 11e6de040 933068800 456187824 227278 227189 name free used name free used overhead 0 3288 scb 0 144 opentable 0 6616 filetable 0 1768 misc 0 272 log 0 16536 temprec 0 19616 blob 0 832 gentcb 0 1632 ostcb 0 2872 tbutility 0 144 sqscb 0 7968 sql 0 72 hashfiletab 0 552 osenv 0 2760 oncheck 0 476813920 sqtcb 0 1904 fragman 0 80 ===========------------- - - - - - - /u/informix11.5-3/bin/onstat -g env 55: IBM Informix Dynamic Server Version 11.50.FC3W1 -- Read-Only (Sec) -- Up 00:18:25 -- 1843200 Kbytes Environment for session 55: Variable Value [values-list] CLIENT_LOCALE en_US.8859-1 DBDELIMITER | DBPATH . [.] [/tmp] DBPRINT lp -s DBTEMP /tmp INFORMIXDIR /u/informix11.5-3 [/u/informix11.5-3] [/usr/informix] INFORMIXSERVER ids_sec02 INFORMIXSQLHOSTS /u/informix11.5-3/etc/sqlhosts INFORMIXSTACKSIZE 128 INFORMIXTERM termcap LC_COLLATE en_US.819 LC_CTYPE en_US.819 LC_MONETARY en_US.819 LC_NUMERIC en_US.819 LC_TIME en_US.819 LKNOTIFY yes LOCKDOWN no NODEFDAC no [no] [no] NO_SUBQF 1 ONCONFIG onconfig_sec02 PATH /usr/sbin:/usr/bin:/usr/local/bin:/usr/sfw/bin:/sbin:/usr/s fw/sparc-sun-solaris2.10/bin/:/usr/local/ssl/bin:/u/inform ix11.5-3/bin SERVER_LOCALE en_US.819 SHELL /bin/bash SINGLELEVEL no SUBQCACHESZ 10 TERM xterm [xterm] [dumb] TERMCAP /etc/termcap TZ US/Pacific Regards, Abrar

OK, I'm going to go out on a limb and suggest that you follow the instructions in the online log: 02:33:01 Action: Please notify IBM Informix Technical Support. Call IBM tech support. You have hit a bug. There may be a workaround or, since you are running 3 fixpacks behind the current release there may be a fixpack available with the bug fix included. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) IIUG Board of Directors (art@iiug.org) See you at the 2010 IIUG Informix Conference April 25-28, 2010 Overland Park (Kansas City), KS www.iiug.org/conf Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, Jan 22, 2010 at 10:28 AM, ABRAR RASHID <mabrar@i2cinc.com> wrote: > Hello Art, > > Thanks for your response. Please find below the required details: > > I hope these would be helpful, however, if you need any further details > then > do let me know. > > $ onstat -V> IBM Informix Dynamic Server Version 11.50.FC3W1 Software Serial Number > AAA#B000000 > $ uname -a> SunOS DB1 5.10 Generic_141414-08 sun4v sparc SUNW,Sun-Fire-T200 > > Command Executed on this Instance: oncheck -cD test_db:table-1 > > Here are the extract from the online.log file > ------------------online.log--------------------------------------------- > 02:32:21 Logical Log 42461 Complete, timestamp: 0x60dde533. > 02:33:01 stack trace for pid 23303 written to > /u/informix11.5-3/tmp/af.4807edc > 02:33:01 Assert Failed: No Exception Handler > 02:33:01 IBM Informix Dynamic Server Version 11.50.FC3W1 > 02:33:01 Who: Session(55, informix@DB1, 23690, 11b13a668) > > Thread(152, oncheck, 11b107680, 11) > > File: mtex.c Line: 491 > 02:33:01 Results: Exception Caught. Type: MT_EX_OS, Context: mem > 02:33:01 Action: Please notify IBM Informix Technical Support. > 02:33:01 See Also: /u/informix11.5-3/tmp/af.4807edc, shmem.4807edc.0 > Assertion Failure Type: CRASH > Host Name: DB1 > Database Server Name: ids_sec02 > Time of failure: Fri Jan 22 02:33:02 PST 2010 > AF file: /u/informix11.5-3/tmp/af.4807edc > Shared memory file: None > System Blocking: OFF > > 02:33:57 mtex.c, line 491, thread 152, proc id 23303, No Exception Handler. > SENDER IS NULL NO MAIL WILL BE SENT > 02:33:57 Fatal error in ADM VP at mt.c:13434 > 02:33:57 Unexpected virtual processor termination, pid = 23303, exit = > 0x100 > > SENDER IS NULL NO MAIL WILL BE SENT > 02:33:58 PANIC: Attempting to bring system down > ------------------online.log--------------------------------------------- > > This is the extract from /u/informix11.5-3/tmp/af.4807edc file > > ===========------------- - - - - - - > /u/informix11.5-3/bin/onstat -g ses 55: > > IBM Informix Dynamic Server Version 11.50.FC3W1 -- Read-Only (Sec) -- Up > 00:18:24 -- 1843200 Kbytes> > session effective #RSAM total used dynamic > id user user tty pid hostname threads memory memory explain > 55 informix - 16 23690 DB1 1 933068800 476880976 off > > tid name rstcb flags curstk status > 152 oncheck 11b107680 ---PRR- 465394207 running- > > Memory pools count 1 > name class addr totalsize freesize #allocfrag #freefrag > 55 V 11e6de040 933068800 456187824 227278 227189 > > name free used name free used > overhead 0 3288 scb 0 144 > opentable 0 6616 filetable 0 1768 > misc 0 272 log 0 16536 > temprec 0 19616 blob 0 832 > gentcb 0 1632 ostcb 0 2872 > tbutility 0 144 sqscb 0 7968 > sql 0 72 hashfiletab 0 552 > osenv 0 2760 oncheck 0 476813920 > sqtcb 0 1904 fragman 0 80 > > ===========------------- - - - - - - > /u/informix11.5-3/bin/onstat -g env 55: > > IBM Informix Dynamic Server Version 11.50.FC3W1 -- Read-Only (Sec) -- Up > 00:18:25 -- 1843200 Kbytes> > Environment for session 55: > > Variable Value [values-list] > CLIENT_LOCALE en_US.8859-1 > DBDELIMITER | > DBPATH . > > [.] > > [/tmp] > DBPRINT lp -s > DBTEMP /tmp > INFORMIXDIR /u/informix11.5-3 > > [/u/informix11.5-3] > > [/usr/informix] > INFORMIXSERVER ids_sec02 > INFORMIXSQLHOSTS /u/informix11.5-3/etc/sqlhosts > INFORMIXSTACKSIZE 128 > INFORMIXTERM termcap > LC_COLLATE en_US.819 > LC_CTYPE en_US.819 > LC_MONETARY en_US.819 > LC_NUMERIC en_US.819 > LC_TIME en_US.819 > LKNOTIFY yes > LOCKDOWN no > NODEFDAC no > > [no] > > [no] > NO_SUBQF 1 > ONCONFIG onconfig_sec02 > PATH /usr/sbin:/usr/bin:/usr/local/bin:/usr/sfw/bin:/sbin:/usr/s > > fw/sparc-sun-solaris2.10/bin/:/usr/local/ssl/bin:/u/inform > > ix11.5-3/bin > SERVER_LOCALE en_US.819 > SHELL /bin/bash > SINGLELEVEL no > SUBQCACHESZ 10 > TERM xterm > > [xterm] > > [dumb] > TERMCAP /etc/termcap > TZ US/Pacific > > Regards, > Abrar > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --00151747b3969a4df7047dc29335