Fast Recovery (CKPT REQ)

Posted in 2014

Topics: Storage & Space Management, Error Codes & Troubleshooting, Platform-Specific Issues

NAVAID ARIF — Fri, 11 Apr 2014 23:07:08 — source: IIUG Forums & Mailing Lists

Hi there, There was a electricity failure in my data center and my informix database server was also down. After restarting the DB server, we are facing following errors in the online.log file and below is the output of (onstat -) command: #onstat - IBM Informix Dynamic Server Version 10.00.UC5 -- Fast Recovery (CKPT REQ) -- Up 00:27:26 --534288 KBytesBlocked:CKPT I would be thankful if anyone can tell me what should be done to make it ONLINE and what went wrong in the database? Below is the extract from online.log file: Sat Apr 12 00:02:47 2014 00:02:47 Event alarms enabled. ALARMPROG = '/informix/ids10/etc/alarmprogram.sh' 00:02:47 Booting Language <c> from module <> 00:02:47 Loading Module <CNULL> 00:02:47 Booting Language <builtin> from module <> 00:02:47 Loading Module <BUILTINNULL> 00:02:52 DR: DRAUTO is 0 (Off) 00:02:52 AIX MP latch code enabled 00:02:53 IBM Informix Dynamic Server Version 10.00.UC5 Software Serial Number AAA#BXXXXXX 00:02:55 IBM Informix Dynamic Server Initialized -- Shared Memory Initialized. 00:02:55 Physical Recovery Started at Page (1:18479). 00:02:55 Physical Recovery Complete: 7 Pages Examined, 7 Pages Restored. 00:02:56 Logical Recovery Started. 00:02:56 10 recovery worker threads will be started. 00:02:56 Assert Warning: I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 00:02:56 IBM Informix Dynamic Server Version 10.00.UC5 00:02:56 Who: Session(7, informix@ttssec, 0, 50229670) Thread(27, xchg_1.0, 5020136c, 4) File: rsbuff.c Line: 5412 00:02:56 Action: Please notify IBM Informix Technical Support. 00:02:56 stack trace for pid 27752 written to /informix/ids10/tmp/af.4033c5f 00:02:56 See Also: /informix/ids10/tmp/af.4033c5f 00:02:56 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 00:02:58 Assert Warning: I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 00:02:58 IBM Informix Dynamic Server Version 10.00.UC5 00:02:58 Who: Session(7, informix@ttssec, 0, 5022a2a0) Thread(27, xchg_1.0, 5020136c, 4) File: rsbuff.c Line: 5412 00:02:58 Action: Please notify IBM Informix Technical Support. 00:02:58 stack trace for pid 27752 written to /informix/ids10/tmp/af.4033c5f 00:02:58 See Also: /informix/ids10/tmp/af.4033c5f 00:02:58 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 00:02:59 Assert Warning: I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 00:02:59 IBM Informix Dynamic Server Version 10.00.UC5 00:02:59 Who: Session(7, informix@appsec, 0, 5022c938) Thread(27, xchg_1.0, 5020136c, 4) File: rsbuff.c Line: 5412 00:02:59 Action: Please notify IBM Informix Technical Support. 00:02:59 stack trace for pid 27752 written to /informix/ids10/tmp/af.4033c5f 00:02:59 See Also: /informix/ids10/tmp/af.4033c5f 00:03:00 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 00:03:01 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 00:03:01 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 00:03:01 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 00:03:01 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 00:03:01 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 . . 02:16:14 Log Record: log = 23658, pos = 0x54bb89c, type = OLDRSAM:HUPAFT(43), trans = 64 02:16:14 Rollforward of log record failed. iserrno = 101 02:16:14 Log Record: log = 23658, pos = 0x54bb89c, type = OLDRSAM:HUPAFT(43), trans = 64 02:16:18 Rollforward of log record failed. iserrno = 101 02:16:18 Log Record: log = 23658, pos = 0x54e0314, type = OLDRSAM:HINSERT(40), trans = 64 02:16:18 Rollforward of log record failed. iserrno = 101 02:16:18 Log Record: log = 23658, pos = 0x54e0314, type = OLDRSAM:HINSERT(40), trans = 64 02:16:18 Rollforward of log record failed. iserrno = 101 02:16:18 Log Record: log = 23658, pos = 0x54c0314, type = OLDRSAM:HINSERT(40), trans = 64 02:16:18 Rollforward of log record failed. iserrno = 101 02:16:18 Log Record: log = 23658, pos = 0x54c0314, type = OLDRSAM:HINSERT(40), trans = 64 02:16:18 Rollforward of log record failed. iserrno = 101 02:16:18 Log Record: log = 23658, pos = 0x54df3ec, type = OLDRSAM:HUPAFT(43), trans = 64 02:16:18 Rollforward of log record failed. iserrno = 101 02:16:18 Log Record: log = 23658, pos = 0x54df3ec, type = OLDRSAM:HUPAFT(43), trans = 64 02:16:27 Rollforward of log record failed. iserrno = 101 02:16:27 Log Record: log = 23658, pos = 0x54c133c, type = OLDRSAM:HINSERT(40), trans = 64 02:16:28 Rollforward of log record failed. iserrno = 101 02:16:28 Log Record: log = 23658, pos = 0x54c133c, type = OLDRSAM:HINSERT(40), trans = 64 02:16:28 Rollforward of log record failed. iserrno = 101 02:16:28 Log Record: log = 23658, pos = 0x54e233c, type = OLDRSAM:HINSERT(40), trans = 64 02:16:28 Rollforward of log record failed. iserrno = 101 02:16:28 Log Record: log = 23658, pos = 0x54e233c, type = OLDRSAM:HINSERT(40), trans = 64 02:16:29 Rollforward of log record failed. iserrno = 101 02:16:29 Log Record: log = 23658, pos = 0x54e133c, type = OLDRSAM:HINSERT(40), trans = 64 Regards, Navaid Arif

↪ replying to NAVAID ARIF

Lloyd S — Sat, 12 Apr 2014 06:26:14 — source: IIUG Forums & Mailing Lists

Naved, The online log points to an error in chunk 2. Are u able to view the file. What are the permission on that file. Any leads on the ad file in tmp directory. Lloyd From: "NAVAID ARIF"navaid.arif@access.net.pk Sent:Sat, 12 Apr 2014 04:37:52 +0530 To: ids@iiug.org Subject: Fast Recovery (CKPT REQ) [32905] Hi there, > There was a electricity failure in my data center and my informix database > server was also down. After restarting the DB server, we are facing following > errors in the online.log file and below is the output of (onstat -) command: > > #onstat - > IBM Informix Dynamic Server Version 10.00.UC5 -- Fast Recovery (CKPT REQ) -- > Up 00:27:26 --534288 KBytes> Blocked:CKPT > > I would be thankful if anyone can tell me what should be done to make it > ONLINE and what went wrong in the database? > Below is the extract from online.log file: > > Sat Apr 12 00:02:47 2014 > > 00:02:47 Event alarms enabled. ALARMPROG = > '/informix/ids10/etc/alarmprogram.sh' > 00:02:47 Booting Language <c> from module <> > 00:02:47 Loading Module <CNULL> > 00:02:47 Booting Language <builtin> from module <> > 00:02:47 Loading Module <BUILTINNULL> > 00:02:52 DR: DRAUTO is 0 (Off) > 00:02:52 AIX MP latch code enabled > 00:02:53 IBM Informix Dynamic Server Version 10.00.UC5 Software Serial Number > AAA#BXXXXXX > 00:02:55 IBM Informix Dynamic Server Initialized -- Shared Memory Initialized. > > 00:02:55 Physical Recovery Started at Page (1:18479). > 00:02:55 Physical Recovery Complete: 7 Pages Examined, 7 Pages Restored. > 00:02:56 Logical Recovery Started. > 00:02:56 10 recovery worker threads will be started. > 00:02:56 Assert Warning: I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = > 9 > 00:02:56 IBM Informix Dynamic Server Version 10.00.UC5 > 00:02:56 Who: Session(7, informix@ttssec, 0, 50229670) > > Thread(27, xchg_1.0, 5020136c, 4) > > File: rsbuff.c Line: 5412 > 00:02:56 Action: Please notify IBM Informix Technical Support. > 00:02:56 stack trace for pid 27752 written to /informix/ids10/tmp/af.4033c5f > 00:02:56 See Also: /informix/ids10/tmp/af.4033c5f > 00:02:56 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 > 00:02:58 Assert Warning: I/O read chunk 2, pagenum 1369756, pagecnt 1 --> > errno = 9 > 00:02:58 IBM Informix Dynamic Server Version 10.00.UC5 > 00:02:58 Who: Session(7, informix@ttssec, 0, 5022a2a0) > > Thread(27, xchg_1.0, 5020136c, 4) > > File: rsbuff.c Line: 5412 > 00:02:58 Action: Please notify IBM Informix Technical Support. > 00:02:58 stack trace for pid 27752 written to /informix/ids10/tmp/af.4033c5f > 00:02:58 See Also: /informix/ids10/tmp/af.4033c5f > 00:02:58 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 > 00:02:59 Assert Warning: I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = > 9 > 00:02:59 IBM Informix Dynamic Server Version 10.00.UC5 > 00:02:59 Who: Session(7, informix@appsec, 0, 5022c938) > > Thread(27, xchg_1.0, 5020136c, 4) > > File: rsbuff.c Line: 5412 > 00:02:59 Action: Please notify IBM Informix Technical Support. > 00:02:59 stack trace for pid 27752 written to /informix/ids10/tmp/af.4033c5f > 00:02:59 See Also: /informix/ids10/tmp/af.4033c5f > 00:03:00 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 > 00:03:01 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 > 00:03:01 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 > 00:03:01 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 > 00:03:01 I/O read chunk 2, pagenum 495, pagecnt 1 --> errno = 9 > 00:03:01 I/O read chunk 2, pagenum 1369756, pagecnt 1 --> errno = 9 > .. > .. > 02:16:14 Log Record: log = 23658, pos = 0x54bb89c, type = OLDRSAM:HUPAFT(43), > trans = 64 > 02:16:14 Rollforward of log record failed. iserrno = 101 > 02:16:14 Log Record: log = 23658, pos = 0x54bb89c, type = OLDRSAM:HUPAFT(43), > trans = 64 > 02:16:18 Rollforward of log record failed. iserrno = 101 > 02:16:18 Log Record: log = 23658, pos = 0x54e0314, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:18 Rollforward of log record failed. iserrno = 101 > 02:16:18 Log Record: log = 23658, pos = 0x54e0314, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:18 Rollforward of log record failed. iserrno = 101 > 02:16:18 Log Record: log = 23658, pos = 0x54c0314, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:18 Rollforward of log record failed. iserrno = 101 > 02:16:18 Log Record: log = 23658, pos = 0x54c0314, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:18 Rollforward of log record failed. iserrno = 101 > 02:16:18 Log Record: log = 23658, pos = 0x54df3ec, type = OLDRSAM:HUPAFT(43), > trans = 64 > 02:16:18 Rollforward of log record failed. iserrno = 101 > 02:16:18 Log Record: log = 23658, pos = 0x54df3ec, type = OLDRSAM:HUPAFT(43), > trans = 64 > 02:16:27 Rollforward of log record failed. iserrno = 101 > 02:16:27 Log Record: log = 23658, pos = 0x54c133c, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:28 Rollforward of log record failed. iserrno = 101 > 02:16:28 Log Record: log = 23658, pos = 0x54c133c, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:28 Rollforward of log record failed. iserrno = 101 > 02:16:28 Log Record: log = 23658, pos = 0x54e233c, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:28 Rollforward of log record failed. iserrno = 101 > 02:16:28 Log Record: log = 23658, pos = 0x54e233c, type = OLDRSAM:HINSERT(40), > trans = 64 > 02:16:29 Rollforward of log record failed. iserrno = 101 > 02:16:29 Log Record: log = 23658, pos = 0x54e133c, type = OLDRSAM:HINSERT(40), > trans = 64 > > Regards, > Navaid Arif > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > >

Related threads

IDS Performance Problem

Fast Recovery (CKPT REQ)

Related threads

Glossary

Your privacy choices