Server Crashed

Posted in 2003

Topics: Error Codes & Troubleshooting, Server Administration, Logging & Checkpoints, Platform-Specific Issues

Vineet Mehrotra — Thu, 26 Jun 2003 02:32:11 -0700 (PDT) — source: Usenet: comp.databases.informix

Informix Dynamic Server 7.31UD2R1 on HP-UX 11.0 Hi All, Our informix instance crashed this morning without giving much information. Here are the excerpts from the message log. 03:34:01 Checkpoint Completed: duration was 0 seconds. 03:34:01 Checkpoint loguniq 47185, logpos 0x291a018 03:34:34 Assert Failed: No Exception Handler 03:34:34 Informix Dynamic Server Version 7.31.UD2R1 03:34:34 Who: Session(0, @, 0, 0) Thread(0, noname, 0, 0) File: mtex.c Line: 450 03:34:34 Results: Exception Caught. Type: MT_EX_OS,Context: mem 03:34:34 Action: Please notify Informix Technical Support. 03:34:34 stack trace for pid 26705 written to /usr/local/log/af.2b019 03:34:34 See Also: /usr/local/log/af.2b019 03:34:34 mtex.c, line 450, thread 0, proc id 26705,No Exception Handler. 03:34:34 invoke_alarm(): /bin/sh -c '/usr/local/dba/bin/inf_alarm_arda_0_1602 5 6 "Internal Subsystem failure: 'MT'" "mtex.c, line 450, thread 0, proc id 26705 , No Exception Handler." ' 03:34:34 invoke_alarm(): mt_exec failed, status 127,errno 0 03:34:34 PANIC: Attempting to bring system down 03:34:34 semctl: errno = 22 Also here is the output of the file af.2b019 03:34:34 Stack for thread: 0 base: 0x0074f158 len: 24576 pc: 0x00000000 tos: 0x007501c0 state: running vp: 0 Out of memory while reading in symbol table of /usr/local/informix/bin/oninit ( 0) 0x00441a08 [/usr/local/informix/bin/oninit] ( 1) 0x0044122c [/usr/local/informix/bin/oninit] ( 2) 0x00442f08 [/usr/local/informix/bin/oninit] ( 3) 0x000aa68c [/usr/local/informix/bin/oninit] Out of memory while reading in symbol table of /usr/lib/libc.2 ( 4) 0xc01f4c20 [/usr/lib/libc.2] ( 5) 0x0044357c [/usr/local/informix/bin/oninit] ( 6) 0x0041fab4 [/usr/local/informix/bin/oninit] ( 7) 0x00421f14 [/usr/local/informix/bin/oninit] ( 8) 0x00422820 [/usr/local/informix/bin/oninit] ( 9) 0x004437e4 [/usr/local/informix/bin/oninit] (10) 0x004438b4 [/usr/local/informix/bin/oninit] (11) 0x00443664 [/usr/local/informix/bin/oninit] (12) 0x00443568 [/usr/local/informix/bin/oninit] base: 0x0074f158 len: 24576 pc: 0x00000000 tos: 0x007501c0 state: running vp: 0 03:34:34 03:34:34 Informix Dynamic Server Version 7.31.UD2R1Software Serial Number AAD #J267574 03:34:34 Assert Failed: No Exception Handler 03:34:34 Who: Session(0, @, 0, 0) Thread(0, noname, 0, 0) File: mtex.c Line: 450 03:34:34 Results: Exception Caught. Type: MT_EX_OS,Context: mem 03:34:34 Action: Please notify Informix Technical Support. 03:34:34 Stack for thread: 0 base: 0x0074f158 len: 24576 pc: 0x00000000 tos: 0x00750900 state: running vp: 0 Out of memory while reading in symbol table of /usr/local/informix/bin/oninit ( 0) 0x00441a08 [/usr/local/informix/bin/oninit] ( 1) 0x0044122c [/usr/local/informix/bin/oninit] ( 2) 0x00440ae4 [/usr/local/informix/bin/oninit] ( 3) 0x00440244 [/usr/local/informix/bin/oninit] ( 4) 0x00442f84 [/usr/local/informix/bin/oninit] ( 5) 0x000aa68c [/usr/local/informix/bin/oninit] Out of memory while reading in symbol table of /usr/lib/libc.2 ( 6) 0xc01f4c20 [/usr/lib/libc.2] ( 7) 0x0044357c [/usr/local/informix/bin/oninit] ( 8) 0x0041fab4 [/usr/local/informix/bin/oninit] ( 9) 0x00421f14 [/usr/local/informix/bin/oninit] (10) 0x00422820 [/usr/local/informix/bin/oninit] (11) 0x004437e4 [/usr/local/informix/bin/oninit] (12) 0x004438b4 [/usr/local/informix/bin/oninit] (13) 0x00443664 [/usr/local/informix/bin/oninit] (14) 0x00443568 [/usr/local/informix/bin/oninit] base: 0x0074f158 len: 24576 pc: 0x00000000 tos: 0x00750900 state: running vp: 0 03:34:34 sh /usr/local/informix/etc/evidence.sh 3 0 /usr/local/log/af.2b019 0 0x0 0 0x7426a8 1537 0 0 0 0 03:34:34 See Also: /usr/local/log/af.2b019 03:34:34 ------------------ End of assertion failure 0 ----------------- Has anybody have any idea about whats going wrong here? I don't have any clue. Thanks, Vineet __________________________________ Do you Yahoo!? SBC Yahoo! DSL - Now only $29.95 per month! http://sbc.yahoo.com sending to informix-list

↪ replying to Vineet Mehrotra

ar — Thu, 26 Jun 2003 16:20:14 +0200 — source: Usenet: comp.databases.informix

Hello, I'm sorry for this response but I'm afraid that the Informix technical support is the only one who could solve this kind of crash. It seems to be a mutex (ressource sharing) problem, but I can't tell you more. Regards AR "Vineet Mehrotra" <vin_us@yahoo.com> a 'crit dans le message de news:bdegv2$n72$1@terabinaries.xmission.com... > > Informix Dynamic Server 7.31UD2R1 on > HP-UX 11.0 > > Hi All, > > Our informix instance crashed this morning without > giving much information. Here are the excerpts from > the message log. > > > 03:34:01 Checkpoint Completed: duration was 0 > seconds. > 03:34:01 Checkpoint loguniq 47185, logpos 0x291a018 > > 03:34:34 Assert Failed: No Exception Handler > 03:34:34 Informix Dynamic Server Version 7.31.UD2R1 > 03:34:34 Who: Session(0, @, 0, 0) > Thread(0, noname, 0, 0) > File: mtex.c Line: 450 > 03:34:34 Results: Exception Caught. Type: MT_EX_OS,> Context: mem > 03:34:34 Action: Please notify Informix Technical > Support. > 03:34:34 stack trace for pid 26705 written to > /usr/local/log/af.2b019 > 03:34:34 See Also: /usr/local/log/af.2b019 > 03:34:34 mtex.c, line 450, thread 0, proc id 26705,> No Exception Handler. > 03:34:34 invoke_alarm(): /bin/sh -c > '/usr/local/dba/bin/inf_alarm_arda_0_1602 5> 6 "Internal Subsystem failure: 'MT'" "mtex.c, line > 450, thread 0, proc id 26705 > , No Exception Handler." ' > 03:34:34 invoke_alarm(): mt_exec failed, status 127,> errno 0 > 03:34:34 PANIC: Attempting to bring system down > 03:34:34 semctl: errno = 22> > > Also here is the output of the file af.2b019 > > > 03:34:34 Stack for thread: 0> > base: 0x0074f158 > len: 24576 > pc: 0x00000000 > tos: 0x007501c0 > state: running > vp: 0 > > Out of memory while reading in symbol table of > /usr/local/informix/bin/oninit > ( 0) 0x00441a08 [/usr/local/informix/bin/oninit] > ( 1) 0x0044122c [/usr/local/informix/bin/oninit] > ( 2) 0x00442f08 [/usr/local/informix/bin/oninit] > ( 3) 0x000aa68c [/usr/local/informix/bin/oninit] > Out of memory while reading in symbol table of > /usr/lib/libc.2 > ( 4) 0xc01f4c20 [/usr/lib/libc.2] > ( 5) 0x0044357c [/usr/local/informix/bin/oninit] > ( 6) 0x0041fab4 [/usr/local/informix/bin/oninit] > ( 7) 0x00421f14 [/usr/local/informix/bin/oninit] > ( 8) 0x00422820 [/usr/local/informix/bin/oninit] > ( 9) 0x004437e4 [/usr/local/informix/bin/oninit] > (10) 0x004438b4 [/usr/local/informix/bin/oninit] > (11) 0x00443664 [/usr/local/informix/bin/oninit] > (12) 0x00443568 [/usr/local/informix/bin/oninit] > > base: 0x0074f158 > len: 24576 > pc: 0x00000000 > tos: 0x007501c0 > state: running > vp: 0 > > > > 03:34:34 > 03:34:34 Informix Dynamic Server Version 7.31.UD2R1> Software Serial Number AAD > #J267574 > > 03:34:34 Assert Failed: No Exception Handler > 03:34:34 Who: Session(0, @, 0, 0) > Thread(0, noname, 0, 0) > File: mtex.c Line: 450 > 03:34:34 Results: Exception Caught. Type: MT_EX_OS,> Context: mem > 03:34:34 Action: Please notify Informix Technical > Support. > > 03:34:34 Stack for thread: 0> > base: 0x0074f158 > len: 24576 > pc: 0x00000000 > tos: 0x00750900 > state: running > vp: 0 > > Out of memory while reading in symbol table of > /usr/local/informix/bin/oninit > ( 0) 0x00441a08 [/usr/local/informix/bin/oninit] > ( 1) 0x0044122c [/usr/local/informix/bin/oninit] > ( 2) 0x00440ae4 [/usr/local/informix/bin/oninit] > ( 3) 0x00440244 [/usr/local/informix/bin/oninit] > ( 4) 0x00442f84 [/usr/local/informix/bin/oninit] > ( 5) 0x000aa68c [/usr/local/informix/bin/oninit] > Out of memory while reading in symbol table of > /usr/lib/libc.2 > ( 6) 0xc01f4c20 [/usr/lib/libc.2] > ( 7) 0x0044357c [/usr/local/informix/bin/oninit] > ( 8) 0x0041fab4 [/usr/local/informix/bin/oninit] > ( 9) 0x00421f14 [/usr/local/informix/bin/oninit] > (10) 0x00422820 [/usr/local/informix/bin/oninit] > (11) 0x004437e4 [/usr/local/informix/bin/oninit] > (12) 0x004438b4 [/usr/local/informix/bin/oninit] > (13) 0x00443664 [/usr/local/informix/bin/oninit] > (14) 0x00443568 [/usr/local/informix/bin/oninit] > > base: 0x0074f158 > len: 24576 > pc: 0x00000000 > tos: 0x00750900 > state: running > vp: 0 > > > > 03:34:34 sh /usr/local/informix/etc/evidence.sh 3 0 > /usr/local/log/af.2b019 0 0> x0 0 0x7426a8 1537 0 0 0 0 > 03:34:34 See Also: /usr/local/log/af.2b019 > 03:34:34 > ------------------ End of assertion failure 0 > -----------------> > > Has anybody have any idea about whats going wrong > here? > I don't have any clue. > > Thanks, > Vineet > > > > __________________________________ > Do you Yahoo!? > SBC Yahoo! DSL - Now only $29.95 per month! > http://sbc.yahoo.com > sending to informix-list

↪ replying to ar

Brice Avila — 26 Jun 2003 14:31:25 -0700 — source: Usenet: comp.databases.informix

I doubt that Informix support is going to be able to help you either. There wasn't enough memory to show a stack trace. Perhaps if you increase kernel memory resources, then the next time this problem happens you'll be able to have a readable stack trace. Good luck. Brice Avila Minneapolis, Minnesota "ar" <bidon@email.com> wrote in message news:<bdeva4$82f$1@news-reader2.wanadoo.fr>... > Hello, > > I'm sorry for this response but I'm afraid that the Informix technical > support is the only one who could solve this kind of crash. > It seems to be a mutex (ressource sharing) problem, but I can't tell you > more. > > Regards > AR > > "Vineet Mehrotra" <vin_us@yahoo.com> a 'crit dans le message de > news:bdegv2$n72$1@terabinaries.xmission.com... > > > > Informix Dynamic Server 7.31UD2R1 on > > HP-UX 11.0 > > > > Hi All, > > > > Our informix instance crashed this morning without > > giving much information. Here are the excerpts from > > the message log. > > > > > > 03:34:01 Checkpoint Completed: duration was 0 > > seconds. > > 03:34:01 Checkpoint loguniq 47185, logpos 0x291a018 > > > > 03:34:34 Assert Failed: No Exception Handler > > 03:34:34 Informix Dynamic Server Version 7.31.UD2R1 > > 03:34:34 Who: Session(0, @, 0, 0) > > Thread(0, noname, 0, 0) > > File: mtex.c Line: 450 > > 03:34:34 Results: Exception Caught. Type: MT_EX_OS,> > Context: mem > > 03:34:34 Action: Please notify Informix Technical > > Support. > > 03:34:34 stack trace for pid 26705 written to > > /usr/local/log/af.2b019 > > 03:34:34 See Also: /usr/local/log/af.2b019 > > 03:34:34 mtex.c, line 450, thread 0, proc id 26705,> > No Exception Handler. > > 03:34:34 invoke_alarm(): /bin/sh -c > > '/usr/local/dba/bin/inf_alarm_arda_0_1602 5> > 6 "Internal Subsystem failure: 'MT'" "mtex.c, line > > 450, thread 0, proc id 26705 > > , No Exception Handler." ' > > 03:34:34 invoke_alarm(): mt_exec failed, status 127,> > errno 0 > > 03:34:34 PANIC: Attempting to bring system down > > 03:34:34 semctl: errno = 22> > > > > > Also here is the output of the file af.2b019 > > > > > > 03:34:34 Stack for thread: 0> > > > base: 0x0074f158 > > len: 24576 > > pc: 0x00000000 > > tos: 0x007501c0 > > state: running > > vp: 0 > > > > Out of memory while reading in symbol table of > > /usr/local/informix/bin/oninit > > ( 0) 0x00441a08 [/usr/local/informix/bin/oninit] > > ( 1) 0x0044122c [/usr/local/informix/bin/oninit] > > ( 2) 0x00442f08 [/usr/local/informix/bin/oninit] > > ( 3) 0x000aa68c [/usr/local/informix/bin/oninit] > > Out of memory while reading in symbol table of > > /usr/lib/libc.2 > > ( 4) 0xc01f4c20 [/usr/lib/libc.2] > > ( 5) 0x0044357c [/usr/local/informix/bin/oninit] > > ( 6) 0x0041fab4 [/usr/local/informix/bin/oninit] > > ( 7) 0x00421f14 [/usr/local/informix/bin/oninit] > > ( 8) 0x00422820 [/usr/local/informix/bin/oninit] > > ( 9) 0x004437e4 [/usr/local/informix/bin/oninit] > > (10) 0x004438b4 [/usr/local/informix/bin/oninit] > > (11) 0x00443664 [/usr/local/informix/bin/oninit] > > (12) 0x00443568 [/usr/local/informix/bin/oninit] > > > > base: 0x0074f158 > > len: 24576 > > pc: 0x00000000 > > tos: 0x007501c0 > > state: running > > vp: 0 > > > > > > > > 03:34:34 > > 03:34:34 Informix Dynamic Server Version 7.31.UD2R1> > Software Serial Number AAD > > #J267574 > > > > 03:34:34 Assert Failed: No Exception Handler > > 03:34:34 Who: Session(0, @, 0, 0) > > Thread(0, noname, 0, 0) > > File: mtex.c Line: 450 > > 03:34:34 Results: Exception Caught. Type: MT_EX_OS,> > Context: mem > > 03:34:34 Action: Please notify Informix Technical > > Support. > > > > 03:34:34 Stack for thread: 0> > > > base: 0x0074f158 > > len: 24576 > > pc: 0x00000000 > > tos: 0x00750900 > > state: running > > vp: 0 > > > > Out of memory while reading in symbol table of > > /usr/local/informix/bin/oninit > > ( 0) 0x00441a08 [/usr/local/informix/bin/oninit] > > ( 1) 0x0044122c [/usr/local/informix/bin/oninit] > > ( 2) 0x00440ae4 [/usr/local/informix/bin/oninit] > > ( 3) 0x00440244 [/usr/local/informix/bin/oninit] > > ( 4) 0x00442f84 [/usr/local/informix/bin/oninit] > > ( 5) 0x000aa68c [/usr/local/informix/bin/oninit] > > Out of memory while reading in symbol table of > > /usr/lib/libc.2 > > ( 6) 0xc01f4c20 [/usr/lib/libc.2] > > ( 7) 0x0044357c [/usr/local/informix/bin/oninit] > > ( 8) 0x0041fab4 [/usr/local/informix/bin/oninit] > > ( 9) 0x00421f14 [/usr/local/informix/bin/oninit] > > (10) 0x00422820 [/usr/local/informix/bin/oninit] > > (11) 0x004437e4 [/usr/local/informix/bin/oninit] > > (12) 0x004438b4 [/usr/local/informix/bin/oninit] > > (13) 0x00443664 [/usr/local/informix/bin/oninit] > > (14) 0x00443568 [/usr/local/informix/bin/oninit] > > > > base: 0x0074f158 > > len: 24576 > > pc: 0x00000000 > > tos: 0x00750900 > > state: running > > vp: 0 > > > > > > > > 03:34:34 sh /usr/local/informix/etc/evidence.sh 3 0 > > /usr/local/log/af.2b019 0 0> > x0 0 0x7426a8 1537 0 0 0 0 > > 03:34:34 See Also: /usr/local/log/af.2b019 > > 03:34:34 > > ------------------ End of assertion failure 0 > > -----------------> > > > > > Has anybody have any idea about whats going wrong > > here? > > I don't have any clue. > > > > Thanks, > > Vineet > > > > > > > > __________________________________ > > Do you Yahoo!? > > SBC Yahoo! DSL - Now only $29.95 per month! > > http://sbc.yahoo.com > > sending to informix-list

Glossary

checkpoint