Re: IDS no longer online, now saying "Primary or Mirror Chunks Offline"
Posted in 2003
I'm going to show my ignorance on this one, but how does one "back out
the logs"? I can roll them forward (although at this point in the
game, the logs are at 100%, except for one, which is at 99%), using
onmode -l, but that does not work. How do I "back out" of a log?
--Anthony
Paul Watson <paul@oninit.com> wrote in message news:<3ECDD96E.D67D9BB7@oninit.com>...
> Looks like you've run out of logs. Start in queiscent and back
> out the logs.
>
> Anthony Presley wrote:
> >
> > Group:
> >
> > I got a call from a remote client that I recently picked up, who
> > conveniently doesn't have an informix support contract, of any kind.
> >
> > Their system went down today, and the database has marked their
> > primary chunk (they have no mirror chunks), dbspace1, as Disabled (D).
> > Using dbaccess, I can get into the sysmaster and sysutils database,
> > but not the primary database (azure).
> >
> > In looking through the logs, at about 14:36 they had a reboot (most
> > likely it wasn't a planned shutdown), and the system has logged:
> >
> > 14:36:01 Event alarms enabled. ALARMPROG =
> > '/opt/informix/etc/log_full.sh'
> > 14:36:07 DR: DRAUTO is 0 (Off)
> > 14:36:07 Requested shared memory segment size rounded from 2644KB to> > 2648KB
> > 14:36:07 Informix Dynamic Server Version 7.30.UC7 Software Serial> > Number AAC#J893194
> > 14:36:07 Informix Dynamic Server Initialized -- Shared Memory> > Initialized.
> > 14:36:07 Physical Recovery Started.
> > 14:36:08 Physical Recovery Complete: 48 Pages Restored.
> > 14:36:08 Logical Recovery Started.
> > 14:36:09 Rollforward of log record failed. iserrno = 126
> > 14:36:09 Log Record: log = 2232236, pos = 9c220c, type =
> > OLDRSAM:HUPDATE(73), trans = 8
> > 14:36:09 Rollforward of log record failed. iserrno = 126
> > 14:36:09 Log Record: log = 2232236, pos = 9c17e8, type =
> > OLDRSAM:HUPDATE(73), trans = 31
> > 14:36:09 Assert Failed: Error during recovery left index> > inconsistent.
> > 14:36:09 Informix Dynamic Server Version 7.30.UC7
> > 14:36:09 Who: Session(11, root@sys3.inventconnect.com, 0, 0)
> > Thread(29, xchg_1.4, 0, 1)
> > File: rskey.c Line: 1180
> > 14:36:09 Results: Index 'azure:"".' is now unusable
> > 14:36:09 Action: Run 'oncheck -cI azure:"".'
> > 14:36:09 See Also: /tmp/af.1d26a8
> > 14:36:09 Rollforward of log record failed. iserrno = 105
> > 14:36:09 Log Record: log = 2232236, pos = 9c1698, type =
> > OLDRSAM:ADDITEM(28), trans = 31
> > 14:36:09 Assert Failed: Error during recovery left index> > inconsistent.
> > 14:36:09 Informix Dynamic Server Version 7.30.UC7
> > 14:36:09 Who: Session(11, root@sys3.inventconnect.com, 0, 0)
> > Thread(29, xchg_1.4, 0, 1)
> > File: rskey.c Line: 1180
> > 14:36:09 Results: Index 'azure:"".' is now unusable
> > 14:36:09 Action: Run 'oncheck -cI azure:"".'
> > 14:36:09 See Also: /tmp/af.1d26a8
> > 14:36:09 Rollforward of log record failed. iserrno = 105
> > 14:36:09 Log Record: log = 2232236, pos = 9c16cc, type =
> > OLDRSAM:ADDITEM(28), trans = 31
> > 14:36:09 Assert Failed: Error during recovery left index> > inconsistent.
> > 14:36:09 Informix Dynamic Server Version 7.30.UC7
> > 14:36:09 Who: Session(11, root@sys3.inventconnect.com, 0, 0)
> > Thread(29, xchg_1.4, 0, 1)
> > File: rskey.c Line: 1180
> > 14:36:09 Results: Index 'azure:"".' is now unusable
> > 14:36:09 Action: Run 'oncheck -cI azure:"".'
> > 14:36:09 See Also: /tmp/af.1d26a8
> > 14:36:09 Rollforward of log record failed. iserrno = 105
> > 14:36:09 Log Record: log = 2232236, pos = 9c1700, type =
> > OLDRSAM:ADDITEM(28), trans = 31
> > 14:36:09 Assert Failed: Error during recovery left index> > inconsistent.
> > 14:36:09 Informix Dynamic Server Version 7.30.UC7
> > 14:36:09 Who: Session(11, root@sys3.inventconnect.com, 0, 0)
> > Thread(29, xchg_1.4, 0, 1)
> > File: rskey.c Line: 1180
> > 14:36:09 Results: Index 'azure:"".' is now unusable
> > 14:36:09 Action: Run 'oncheck -cI azure:"".'
> > 14:36:09 See Also: /tmp/af.1d26a8
> > 14:36:09 Rollforward of log record failed. iserrno = 105
> > 14:36:09 Log Record: log = 2232236, pos = 9c1734, type =
> > OLDRSAM:ADDITEM(28), trans = 31
> > 14:36:09 Assert Failed: Error during recovery left index> > inconsistent.
> > 14:36:09 Informix Dynamic Server Version 7.30.UC7
> > 14:36:09 Who: Session(11, root@sys3.inventconnect.com, 0, 0)
> > Thread(29, xchg_1.4, 0, 1)
> > File: rskey.c Line: 1180
> > 14:36:09 Results: Index 'azure:"".' is now unusable
> > 14:36:09 Action: Run 'oncheck -cI azure:"".'
> > 14:36:09 See Also: /tmp/af.1d26a8
> > 14:36:09 Rollforward of log record failed. iserrno = 105
> > 14:36:09 Log Record: log = 2232236, pos = 9c1768, type =
> > OLDRSAM:ADDITEM(28), trans = 31
> > 14:36:09 Assert Failed: Chunk 2 is being taken OFFLINE.
> > 14:36:09 Informix Dynamic Server Version 7.30.UC7
> > 14:36:09 Who: Session(11, root@sys3.inventconnect.com, 0, 0)
> > Thread(27, xchg_1.2, 0, 1)
> > File: rsmirror.c Line: 1772
> > 14:36:09 Results: Dynamic Server will block at next checkpoint
> > 14:36:09 Action: Shutdown (onmode -k) or override (onmode -O)
> > 14:36:09 See Also: /tmp/af.1b26a7
> > 14:36:09 Rollforward of log record failed. iserrno = 101
> > 14:36:09 Log Record: log = 2232236, pos = 9c23ec, type =
> > OLDRSAM:ADDITEM(28), trans = 8
> > 14:36:09 Assert Failed: Chunk 2 is being taken OFFLINE.
> > 14:36:09 Informix Dynamic Server Version 7.30.UC7
> > 14:36:09 Who: Session(11, root@sys3.inventconnect.com, 0, 0)
> > Thread(28, xchg_1.3, 0, 1)
> > File: rsmirror.c Line: 1772
> > 14:36:09 Results: Dynamic Server will block at next checkpoint
> > 14:36:09 Action: Shutdown (onmode -k) or override (onmode -O)
> > 14:36:09 See Also: /tmp/af.1c26a8
> > 14:36:09 Rollforward of log record failed. iserrno = 101
> > 14:36:09 Log Record: log = 2232236, pos = 9c20f0, type =
> > OLDRSAM:ADDITEM(28), trans = 31
> > 14:36:09 Rollforward of log record failed. iserrno = 101
> > 14:36:09 Log Record: log = 2232236, pos = 9c2124, type =
> > OLDRSAM:DELITEM(29), trans = 31
> > 14:36:09 Rollforward of log record failed. iserrno = 101
> > 14:36:09 Log Record: log = 2232236, pos = 9c2158, type =
> > OLDRSAM:ADDITEM(28), trans = 31
> > 14:36:09 Rollforward of log record failed. iserrno = 101
> > 14:36:09 Log Record: log = 2232236, pos = 9c218c, type =
> > OLDRSAM:DELITEM(29), trans = 31
> > 14:36:10 Logical Recovery Complete.> > 13 Committed, 0 Rolled Back, 0 Open, 0 Bad Locks
> >
> > 14:36:10 Dataskip is now OFF for all dbspaces14:36:11 On-Line Mode
> > 14:36:11 Checkpoint blocked by down space, waiting for override or> > shutdown
> >
> > I've tried to run oncheck -cI azure:"", and I get the error "ISAM
> > error: Primary and Mirror chunks are bad"
> >
> > Running oncheck -cRe