IDS 7.31 UD5 data(base) recovery
An IDS 7.31.UD5 instance was stuck in Fast Recovery ("Waiting for Next Logical Log File to be Freed", Blocked:CKPT), with no level-0 archive and no log backups, apparently due to a long transaction rollback. Suggestions included setting LTAPEDEV to a real /dev/null (a symlink isn't enough) rather than a tape path, checking LTXHWM/LTXEHWM and onstat -lx/-d, calling IBM support to add logs or zero the transaction, and Art Kagel's warning not to shut the engine down first. The poster had already shut it down and reported partially recovering data via oncheck; no full resolution is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
hi,
i'm looking for any solution which allow me to get database server
online or export data from dbspace
the state is as follows:
# onstat -Informix Dynamic Server Version 7.31.UD5 -- Fast Recovery (CKPT
REQ) -- Up 22:31:46 -- 98304 Kbytes
Blocked:CKPT
# ontape -aPerforming automatic backup of logical logs.
Please mount tape 1 on /usr/informix/ltapedev and press Return to
continue ...
Logbackup failed - Logical Log File not found.
This tape contains no log data
Program over.
online.log
10:37:09 Informix Dynamic Server Initialized -- Shared MemoryInitialized.
10:37:09 Physical Recovery Started.
10:37:11 Physical Recovery Complete: 596 Pages Restored.
10:37:11 Logical Recovery Started.
10:37:15 Waiting for Next Logical Log File to be Freed
TAPEDEV=/dev/null
LTAPEDEV=/dev/null
logical log files are in rootdbs
# Root Dbspace Configuration
ROOTNAME rootdbs # Root dbspace nameROOTPATH /usr/informix/online_root # Path for device
containing root dbspace
ROOTOFFSET 0 # Offset of root dbspace into device
(Kbytes)
ROOTSIZE 200000 # Size of root dbspace (Kbytes)
# Disk Mirroring Configuration Parameters
MIRROR 0 # Mirroring flag (Yes = 1, No = 0)
MIRRORPATH # Path for device containing mirroredroot
MIRROROFFSET 0 # Offset into mirrored device (Kbytes)
# Physical Log Configuration
PHYSDBS rootdbs # Location (dbspace) of physical log
PHYSFILE 3000 # Physical log file size (Kbytes)
# Logical Log Configuration
LOGFILES 30 # Number of logical log files
LOGSIZE 5000 # Logical log size (Kbytes)
no 0-level archive
no 1-level archive
no logical logs archive
no possibility of restore any dbspaces from backup
is there any way to :
- bring server online ( e.g. manually edit rootdbs raw device )
- export database from non root dbspace
- search for table data manually in dbspace raw device
thx for any help
marcin tomczyk
Hello Marcin,
when i read this:
> # ontape -a> Performing automatic backup of logical logs.
> Please mount tape 1 on /usr/informix/ltapedev and press Return to
it sounds like that your LTAPEDEV is set to /usr/informix/ltapedev
in your onconfig file.
if that ist the case you can set it to /dev/null.
eq in $INFORMIXDIR/etc/$ONCONFIG search for LTAPEDEV and set.
LTAPEDEV /dev/null
Then restart the database, it should come online.
Having set that you could also suffer from a long trx rollback biting
in it's tail.
Post a onstat -c eq your onconfig file (am looking for LTXHWM, LTXEHWM
and a onstat -lx so we can determine if you are having a long trx
rollback
an onstat -d is also helpfull to see if there are any dbspaces/chunks
down....
If you are running into a long trx you need to call Ibm/informix tech
support
to have them add logs or zero the trx so your engine will come online.
Hope it helps
Superboer
On 18 apr, 11:48, Marcin Tomczyk <malta...@gmail.com> wrote:
> hi,
>
> i'm looking for any solution which allow me to get database server
> online or export data from dbspace
>
> the state is as follows:
>
> # onstat -> Informix Dynamic Server Version 7.31.UD5 -- Fast Recovery (CKPT
> REQ) -- Up 22:31:46 -- 98304 Kbytes
> Blocked:CKPT
>
> # ontape -a> Performing automatic backup of logical logs.
> Please mount tape 1 on /usr/informix/ltapedev and press Return to
> continue ...
> Logbackup failed - Logical Log File not found.
> This tape contains no log data
> Program over.
>
> online.log
> 10:37:09 Informix Dynamic Server Initialized -- Shared Memory> Initialized.
> 10:37:09 Physical Recovery Started.
> 10:37:11 Physical Recovery Complete: 596 Pages Restored.
> 10:37:11 Logical Recovery Started.
> 10:37:15 Waiting for Next Logical Log File to be Freed>
> TAPEDEV=/dev/null
> LTAPEDEV=/dev/null
>
> logical log files are in rootdbs
>
> # Root Dbspace Configuration
>
> ROOTNAME rootdbs # Root dbspace name> ROOTPATH /usr/informix/online_root # Path for device
> containing root dbspace
> ROOTOFFSET 0 # Offset of root dbspace into device
> (Kbytes)
> ROOTSIZE 200000 # Size of root dbspace (Kbytes)>
> # Disk Mirroring Configuration Parameters
>
> MIRROR 0 # Mirroring flag (Yes = 1, No = 0)
> MIRRORPATH # Path for device containing mirrored> root
> MIRROROFFSET 0 # Offset into mirrored device (Kbytes)>
> # Physical Log Configuration
>
> PHYSDBS rootdbs # Location (dbspace) of physical log
> PHYSFILE 3000 # Physical log file size (Kbytes)>
> # Logical Log Configuration
>
> LOGFILES 30 # Number of logical log files
> LOGSIZE 5000 # Logical log size (Kbytes)>
> no 0-level archive
> no 1-level archive
> no logical logs archive
> no possibility of restore any dbspaces from backup
>
> is there any way to :
> - bring server online ( e.g. manually edit rootdbs raw device )
> - export database from non root dbspace
> - search for table data manually in dbspace raw device
>
> thx for any help
> marcin tomczyk
I said this to Marcin privately, now let me say it publicly: DO NOT SHUT
DOWN THE SERVER until someone from IBM or someone else with advanced skills
looks at your system unless you can successfully take a level 0 archive
first. Doing so may work out fine, but it may not. This message:
Logbackup failed - Logical Log File not found.
Has me worried about the viability of your chunks and the engine's current
active open files may be the only link to the data that's left.
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other organization with which I am associated either explicitly,
implicitly, or by inference. Neither do those opinions reflect those of
other individuals affiliated with any entity with which I am affiliated nor
those of the entities themselves.
On Wed, Apr 18, 2012 at 1:13 PM, Superboer <superboer7@t-online.de> wrote:
> Hello Marcin,
>
>
> when i read this:
>
> > # ontape -a> > Performing automatic backup of logical logs.
> > Please mount tape 1 on /usr/informix/ltapedev and press Return to
>
>
> it sounds like that your LTAPEDEV is set to /usr/informix/ltapedev
> in your onconfig file.
>
> if that ist the case you can set it to /dev/null.
>
>
> eq in $INFORMIXDIR/etc/$ONCONFIG search for LTAPEDEV and set.
>
> LTAPEDEV /dev/null>
> Then restart the database, it should come online.
>
> Having set that you could also suffer from a long trx rollback biting
> in it's tail.
>
> Post a onstat -c eq your onconfig file (am looking for LTXHWM, LTXEHWM
> and a onstat -lx so we can determine if you are having a long trx
> rollback
> an onstat -d is also helpfull to see if there are any dbspaces/chunks
> down....
> If you are running into a long trx you need to call Ibm/informix tech
> support
> to have them add logs or zero the trx so your engine will come online.
>
> Hope it helps
>
> Superboer
>
>
> On 18 apr, 11:48, Marcin Tomczyk <malta...@gmail.com> wrote:
> > hi,
> >
> > i'm looking for any solution which allow me to get database server
> > online or export data from dbspace
> >
> > the state is as follows:
> >
> > # onstat -> > Informix Dynamic Server Version 7.31.UD5 -- Fast Recovery (CKPT
> > REQ) -- Up 22:31:46 -- 98304 Kbytes
> > Blocked:CKPT
> >
> > # ontape -a> > Performing automatic backup of logical logs.
> > Please mount tape 1 on /usr/informix/ltapedev and press Return to
> > continue ...
> > Logbackup failed - Logical Log File not found.
> > This tape contains no log data
> > Program over.
> >
> > online.log
> > 10:37:09 Informix Dynamic Server Initialized -- Shared Memory> > Initialized.
> > 10:37:09 Physical Recovery Started.
> > 10:37:11 Physical Recovery Complete: 596 Pages Restored.
> > 10:37:11 Logical Recovery Started.
> > 10:37:15 Waiting for Next Logical Log File to be Freed> >
> > TAPEDEV=/dev/null
> > LTAPEDEV=/dev/null
> >
> > logical log files are in rootdbs
> >
> > # Root Dbspace Configuration
> >
> > ROOTNAME rootdbs # Root dbspace name> > ROOTPATH /usr/informix/online_root # Path for device
> > containing root dbspace
> > ROOTOFFSET 0 # Offset of root dbspace into device
> > (Kbytes)
> > ROOTSIZE 200000 # Size of root dbspace (Kbytes)> >
> > # Disk Mirroring Configuration Parameters
> >
> > MIRROR 0 # Mirroring flag (Yes = 1, No = 0)
> > MIRRORPATH # Path for device containing mirrored> > root
> > MIRROROFFSET 0 # Offset into mirrored device (Kbytes)> >
> > # Physical Log Configuration
> >
> > PHYSDBS rootdbs # Location (dbspace) of physical log
> > PHYSFILE 3000 # Physical log file size (Kbytes)> >
> > # Logical Log Configuration
> >
> > LOGFILES 30 # Number of logical log files
> > LOGSIZE 5000 # Logical log size (Kbytes)> >
> > no 0-level archive
> > no 1-level archive
> > no logical logs archive
> > no possibility of restore any dbspaces from backup
> >
> > is there any way to :
> > - bring server online ( e.g. manually edit rootdbs raw device )
> > - export database from non root dbspace
> > - search for table data manually in dbspace raw device
> >
> > thx for any help
> > marcin tomczyk
>
> _______________________________________________
> Informix-list mailing list
> Informix-list@iiug.org
> http://www.iiug.org/mailman/listinfo/informix-list
>
Informix Dynamic Server Version 7.31.UD5 I understood that IDS 7.31 entered End Of Support in September 2009 ( 2 and a half years ago) I hope this is not a business critical system
It did, however, you can pay for extended support. You pay through the nose, but you can get it. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Apr 18, 2012 at 3:04 PM, scottishpoet <dryburghj@yahoo.com> wrote: > Informix Dynamic Server Version 7.31.UD5 > > I understood that IDS 7.31 entered End Of Support in September 2009 > ( 2 and a half years ago) > > I hope this is not a business critical system > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list >
On 18 Kwi, 19:13, Superboer <superbo...@t-online.de> wrote:
> Hello Marcin,
>
> it sounds like that your LTAPEDEV is set to /usr/informix/ltapedev
> in your onconfig file.
>
> if that ist the case you can set it to /dev/null.
>
> eq in $INFORMIXDIR/etc/$ONCONFIG search for LTAPEDEV and set.
>
> LTAPEDEV /dev/null
ltapedev is a symbolic link to /dev/null
>
> Having set that you could also suffer from a long trx rollback biting
> in it's tail.
yes - it was probably bad long transaction aborting
>
> Post a onstat -c eq your onconfig file (am looking for LTXHWM, LTXEHWM
standard values: 50,60
i've tried to make them higher to let the long transaction be rolled
back but with no success.
> an onstat -d is also helpfull to see if there are any dbspaces/chunks
> down....
chunks are ok.
>
> Hope it helps
>
> Superboer
>
thx for help
marcin
On 18 Kwi, 19:50, Art Kagel <art.ka...@gmail.com> wrote:
> I said this to Marcin privately, now let me say it publicly: DO NOT SHUT
> DOWN THE SERVER until someone from IBM or someone else with advanced skills
> looks at your system unless you can successfully take a level 0 archive
i knew that shutting down could be a problem but now it's too late :/
i really appreciate your answer
in this moment i'm doing some successfull data export using 'oncheck'
i send you details (private)
marcin
Hello Marcin,
ltapedev is a symbolic link to /dev/null
that is no good, when LTAPEDEV is set to something else then /dev/
null, the engine
thinks it needs to backup logs.
->>> DO NOT SHUT
> > DOWN THE SERVER until someone from IBM or someone else with advanced skills
> > looks at your system unless you can successfully take a level 0 archive
hmmm when it waits for a log to be free, one is for sure not able to
create a lvl 0 archive.!!
one thing comes to my mind why i would not shut the engine, or be
afraid to do so
and that is that nasty phys log bug which causes phys recovery into a
loop thing, which should not be an issue in your version though!!!
Do not know how big your instance is, you could hmmm afwul dd your
chunks away
but first do a omode -B to flush any dirty buffers, then do the dd to
be safe.
I still did not see your onstat -lx, it still could very well be that
changing LTAPEDEV to realy /dev/null helps
you out of this issue.
Superboer.
BTW sorry that i did not responce sooner. having not much time left
for stuff like this....
On 18 apr, 22:07, Marcin Tomczyk <malta...@gmail.com> wrote:
> On 18 Kwi, 19:50, Art Kagel <art.ka...@gmail.com> wrote:> I said this to Marcin privately, now let me say it publicly: DO NOT SHUT
> > DOWN THE SERVER until someone from IBM or someone else with advanced skills
> > looks at your system unless you can successfully take a level 0 archive
>
> i knew that shutting down could be a problem but now it's too late :/
>
> i really appreciate your answer
> in this moment i'm doing some successfull data export using 'oncheck'
> i send you details (private)
>
> marcin