Rollforward of log record failed. iserrno = 126

Hi Folks, PLEASE HELP !!!!! We have IDS 9.30 UC3 on RHL 7.3 (Kernel - 2.4.18-27.7.xbigmem). Platform - Dell PE6600 - Quard Xeon, dual channel, 8GB RAM - HT. Channel-A has 8 72 GB mirror & stripped and Channel-B has got 8 36 GB mirror & stripped. All of sudden, production server has rebooted at its own. When I tried to bring live instance on-line, I notice an assert failure in informix log file as below: Pl. note - Chunk 6 is our main data dbspace. Unfortunately, IBM Informix support is not available after 5:00 PM here, Can some one pl. advise if there is any way to bring engine on-line without performing COLD RESTORE. I did some research and found this is a known bug (155036) with IDS 9.30 UC4 If this a known bug, does anybody knows the fix so that this situation could be avoided for future ? TIA. Hari -------------------------------------------------------------------------------------------- ..... ..... 17:37:28 Loading Module <BUILTINNULL> 17:37:39 Dynamically allocated new message shared memory segment (size 3220KB) 17:37:39 Informix Dynamic Server Version 9.30.UC3 Software SerialNumber AAD#J118716 17:37:40 Informix Dynamic Server Initialized -- Shared MemoryInitialized. 17:37:40 Physical Recovery Started at Page(2:100646). 17:37:40 Physical Recovery Complete: 1174 Pages Examined 1170 Pages Restored. 17:37:40 Logical Recovery Started. 17:37:40 10 recovery worker threads will be started. 17:37:41 Rollforward of log record failed. iserrno = 126 17:37:41 Log Record: log = 879, pos = 0x1072068, type = OLDRSAM:RDELETE(45), trans = 639 17:37:42 Assert Failed: Chunk 6 is being taken OFFLINE. 17:37:42 Informix Dynamic Server Version 9.30.UC3 17:37:42 Who: Session(16, informix@koa.whittlesea.vic.gov.au, 0,2038718992) Thread(173, xchg_1.3, 79816618, 3) File: rsmirror.c Line: 1766 17:37:42 Results: Dynamic Server must abort 17:37:42 Action: Reinitialize shared memory 17:37:42 stack trace for pid 1475 written to /tmp/af.4958545 17:37:42 See Also: /tmp/af.4958545 17:37:42 Chunk 6 is being taken OFFLINE. 17:37:42 Assert Failed: INFORMIX-OnLine Must ABORT Critical media failure. 17:37:42 Informix Dynamic Server Version 9.30.UC3 17:37:42 Who: Session(16, informix@koa.whittlesea.vic.gov.au, 0,2038718992) Thread(173, xchg_1.3, 79816618, 3) File: rsmirror.c Line: 1770 17:37:42 stack trace for pid 1475 written to /tmp/af.4958545 17:37:42 See Also: /tmp/af.4958545 17:37:45 rsmirror.c, line 1770, thread 173, proc id 1475,INFORMIX-OnLine Must ABORT Critical media failure.. 17:37:45 Fatal error in ADM VP at mt.c:12008 17:37:45 Unexpected virtual processor termination, pid = 1475, exit =0x100 17:37:45 PANIC: Attempting to bring system down 17:37:45 semctl: errno = 22 ----------------------------------------------------------------------------------------- /usr/informix/bin/onstat -d: Informix Dynamic Server Version 9.30.UC3 -- Fast Recovery -- Up 00:00:16 -- 1366176 Kbytes Dbspaces address number flags fchunk nchunks flags owner name 788a07d8 1 0x1 1 1 N informix rootdbs 7986a018 2 0x1 2 1 N informix plogdbs 7986a168 3 0x1 3 1 N informix llogdbs 7986a2b8 4 0x2001 4 1 N T informix tmpndbs 7986a408 5 0x2001 5 1 N T informix tmptdbs 7986a558 6 0x1 6 16 N informix maindbs 7986a6a8 7 0x8001 22 1 N S informix sndqdbs 7 active, 2047 maximum Chunks address chk/dbs offset size free bpages flags pathname 788a0928 1 1 0 256000 198868 PO- /IFMXDATA/authlive/rootdbs 798430a0 2 2 0 256000 5947 PO- /IFMXDATA/authlive/plogdbs 79843210 3 3 0 512000 69579 PO- /IFMXDATA/authlive/llogdbs 79843380 4 4 0 1024000 1023947 PO- /IFMXDATA/authlive/tmpndbs 798434f0 5 5 0 1024000 1023947 PO- /IFMXDATA/authlive/tmptdbs 79843660 6 6 0 1024000 0 PD- /IFMXDATA/authlive/maindbs01 798437d0 7 6 0 1024000 0 PO- /IFMXDATA/authlive/maindbs02 79843940 8 6 0 1024000 51 PO- /IFMXDATA/authlive/maindbs03 79843ab0 9 6 0 1024000 17 PO- /IFMXDATA/authlive/maindbs04 79843c20 10 6 0 1024000 4 PO- /IFMXDATA/authlive/maindbs05 79843d90 11 6 0 1024000 2 PO- /IFMXDATA/authlive/maindbs06 79869018 12 6 0 1024000 8 PO- /IFMXDATA/authlive/maindbs07 79869188 13 6 0 1024000 12 PO- /IFMXDATA/authlive/maindbs08 798692f8 14 6 0 1024000 65279 PO- /IFMXDATA/authlive/maindbs09 79869468 15 6 0 1024000 540241 PO- /IFMXDATA/authlive/maindbs10 798695d8 16 6 0 1024000 1023997 PO- /IFMXDATA/authlive/maindbs11 79869748 17 6 0 1024000 1023997 PO- /IFMXDATA/authlive/maindbs12 798698b8 18 6 0 1024000 1023997 PO- /IFMXDATA/authlive/maindbs13 79869a28 19 6 0 1024000 1023997 PO- /IFMXDATA/authlive/maindbs14 79869b98 20 6 0 1024000 1023997 PO- /IFMXDATA/authlive/maindbs15 79869d08 21 6 0 1024000 1023997 PO- /IFMXDATA/authlive/maindbs16 79869e78 22 7 0 1048575 -1 -1 POS /IFMXDATA/authlive/sndqdbs 22 active, 2047 maximum --------------------------------------------------------------------- [Live:informix->authprog] onstat -c | more Informix Dynamic Server Version 9.30.UC3 -- Fast Recovery -- Up 00:05:53 -- 1366176 Kbytes Configuration File: /usr/informix/etc/onconfig.authlive #************************************************************************** # # INFORMIX SOFTWARE, INC. # # Title: onconfig.authlive.normalrunning # Description: Informix Dynamic Server Configuration Parameters # Use this as the base configuration file for a quad # hyper-threaded P4 server with 2 gig or more of ram # #************************************************************************** # Root Dbspace Configuration ROOTNAME rootdbs # Root dbspace nameROOTPATH /IFMXDATA/authlive/rootdbs # Path for device containing root dbspace ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes) ROOTSIZE 512000 # Size of root dbspace (Kbytes) # Disk Mirroring Configuration Parameters MIRROR 0 # Mirroring flag (Yes = 1, No = 0) MIRRORPATH # Path for device containing mirroredroot MIRROROFFSET 0 # Offset into mirrored device (Kbytes)