oninit fails reporting problem with DBSPACETEMP
Posted in 2009
After an ontape STDIO backup and cold restore on IDS 11.50 (RHEL 5.3), the user's scripted restore sequence produced "Invalid (non-existent/blobspace/disabled) dbspace listed in DBSPACETEMP" warnings for tempdbs1/tempdbs2 at startup, even though the spaces existed. Art Kagel stressed that onmode -m must be run after a restore to complete fast recovery; others noted the -t STDIO syntax. The real cause was the script shutting down too soon after onmode -m. Adding a wait loop that polls onstat's return code (5 = online) before onmode -ky and oninit -v fixed it.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Backup & Restore, Storage & Space Management, Server Administration
Hi,
I am using IDS 11.5 in RHEL 5.3
I am trying to do backup/restore using the STDIO
For backup, i did "ontape -s -L 0 > /dbspace/bkup"
For restore, "onmode -ky; cat /dbspace/bkup | ontape -r"
After that doing the warm restore, the database comes in shared mode as oninit
-r. So, i did onmode -kyv and then oninit -v.
But the database didnt come up reporting the priblem as "Invalid dbspace
listed in DBSPACETEMP" I have tempdbs1 and tempdbs2 as my DBSPACETEMP.
Any suggestion why the system fails with DBSPACES?
Thanks,
Lakki
GOOGLE is your friend! I must have posted this about a hundred times over
the past 16 years - twice in just the last month. Everyone make a note:
After a restore completes, you MUST bring the engine to FULL ONLINE mode by
running onmode -m BEFORE you shut it down with onmode -ky. This runs the
engine through it's final fast recovery code that completes the restore
process by rolling back transactions that were incomplete when either the
archive itself of the last restored logical log was backed up. Otherwise
the chunks will be left in an inconsistent state (their state flags will
tell the engine that they are not usable) and you will not be able to
restart the engine. Your options at this point are:
1. Redo the restore from scratch and complete it properly.
2. Call IBM to dial in and force your chunks online (not recommended
because the fast recovery will not have been completed and there will be
partial transactions leaving inconsistent data).
3. Someone informed me recently that in 11.10+ you can run oninit -r
again to start the engine and then do the onmode -m to complete fast
recovery. I've never tried that myself.
Art
On Mon, Mar 16, 2009 at 11:44 AM, LAKSHMI DEVI PALANISSAMY <
lakshmidevip@hcl.in> wrote:
> Hi,
>
> I am using IDS 11.5 in RHEL 5.3
>
> I am trying to do backup/restore using the STDIO
> For backup, i did "ontape -s -L 0 > /dbspace/bkup"
> For restore, "onmode -ky; cat /dbspace/bkup | ontape -r"
> After that doing the warm restore, the database comes in shared mode as
> oninit
> -r. So, i did onmode -kyv and then oninit -v.>
> But the database didnt come up reporting the priblem as "Invalid dbspace
> listed in DBSPACETEMP" I have tempdbs1 and tempdbs2 as my DBSPACETEMP.
>
> Any suggestion why the system fails with DBSPACES?
>
> Thanks,
> Lakki
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
--001636427491f4ccc704653ea964
Hi Art,
Thanks for your comments!!
I followed the below steps for the restore operation:
1. Bring down the database to Quiscent mode -> onmode -kyv
2. Perform restore operation -> cat bkup_contents | ontape -r
3. onmode -m
4. onmode -ky
5. oninit -v
Still i am able to see the same warning again!
But inspite of the warning, the database comes up proplerly.
Is it advisable to ignore the warning and go ahead???
-Lakshmi
Lakshmi-
IDS list is not threaded, so please include what you are
responding to, so we're not all lost reading your post. I found your
first post, so I can answer.
> -----Original Message-----
> From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
> LAKSHMI DEVI PALANISSAMY
> Sent: Monday, March 16, 2009 2:34 PM
> To: ids@iiug.org
> Subject: Re: oninit fails reporting problem with DBSPACETEM [15167]
>
> Hi Art,
>
> Thanks for your comments!!
>
> I followed the below steps for the restore operation:
> 1. Bring down the database to Quiscent mode -> onmode -kyv
onmode -k brings the engine down entirely, so you are doing a "cold"restore. Onmode -s sets the instance to quiescent mode.
Please provide these for a diagnosis: your entry for DBSPACETEMP in your
onconfig, the output from onstat -d and the output from oninit -v after
your restore. Having an "Invalid dbspace listed in DBSPACETEMP" should
not be a fatal situation. Is the engine dead after the restore, or does
it just drop the error message in the online log and continue?
--EEM
> 2. Perform restore operation -> cat bkup_contents | ontape -r
> 3. onmode -m
> 4. onmode -ky
> 5. oninit -v
>
> Still i am able to see the same warning again!
> But inspite of the warning, the database comes up proplerly.
> Is it advisable to ignore the warning and go ahead???
>
> -Lakshmi
>
>
>
************************************************************************
**
> *****
> Forum Note: Use "Reply" to post a response in the discussion forum.
That is incorrect syntax for using ontape with STDIO.
Correct syntax would be:
ontape -s -L -0 -t STDIO > /backup
cat /backup | ontape -r -t STDIOHTH
Hrvoje
LAKSHMI DEVI PALANISSAMY wrote:
> Hi,
>
> I am using IDS 11.5 in RHEL 5.3
>
> I am trying to do backup/restore using the STDIO
> For backup, i did "ontape -s -L 0 > /dbspace/bkup"
> For restore, "onmode -ky; cat /dbspace/bkup | ontape -r"
> After that doing the warm restore, the database comes in shared mode as
oninit
> -r. So, i did onmode -kyv and then oninit -v.>
> But the database didnt come up reporting the priblem as "Invalid dbspace
> listed in DBSPACETEMP" I have tempdbs1 and tempdbs2 as my DBSPACETEMP.
>
> Any suggestion why the system fails with DBSPACES?
>
> Thanks,
> Lakki
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
>
When do you see the DBTEMPSPACE warning? At the time that ontape starts the
oninit -r or when you later run the oninit -v? If it's the former, that'sjust because those dbspaces don't yet exist at the time the restore begins,
so you can ignore the warning with no harm. If at the final startup, I
don't know. If the dbspaces tempdbs1 and tempdbs2 do indeed exist then you
should not be getting such a warning.
Art
On Mon, Mar 16, 2009 at 3:33 PM, LAKSHMI DEVI PALANISSAMY <
lakshmidevip@hcl.in> wrote:
> Hi Art,
>
> Thanks for your comments!!
>
> I followed the below steps for the restore operation:
> 1. Bring down the database to Quiscent mode -> onmode -kyv
> 2. Perform restore operation -> cat bkup_contents | ontape -r
> 3. onmode -m
> 4. onmode -ky
> 5. oninit -v
>
> Still i am able to see the same warning again!
> But inspite of the warning, the database comes up proplerly.
> Is it advisable to ignore the warning and go ahead???
>
> -Lakshmi
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
--001636aa2c14bd822e0465444d09
The warning message occurs only when bringing up the database (oninit -v)
Also the mentioned dbspace do exist when the database is coming up!!
But in my ONCONFIG file, i have mentioned the tape device as STDIO
DBSPACETEMP are tempdbs1 and tempdbs2
Output of onstat -d:
IBM Informix Dynamic Server Version 11.50.UC3 -- On-Line -- Up 00:00:33 --1091824 Kbytes
Dbspaces
address number flags fchunk nchunks pgsize flags owner name
4a09a808 1 0x40001 1 1 2048 N B informix rootdbs
4a09ac18 2 0x40001 2 1 2048 N B informix dbspace1
4a09ad78 3 0x40001 3 1 2048 N B informix traindb
4aa39018 4 0x40001 4 1 2048 N B informix archdb
4aa39178 5 0x42001 5 1 2048 N TB informix tempdbs1
4aa392d8 6 0x42001 6 1 2048 N TB informix tempdbs2
4aa39438 7 0x40001 7 1 2048 N B informix phylogdbs
4aa39598 8 0x40001 8 1 2048 N B informix llogdbs
8 active, 2047 maximum
Chunks
address chunk/dbs offset size free bpages flags pathname
4a09a968 1 1 0 1000000 993665 PO-B /u/informix/space/rootdbs
4aa396f8 2 2 0 5120000 5117636 PO-B /u/informix/space/dbspace1
4aa398c8 3 3 0 512000 509640 PO-B /u/informix/space/traindb
4aa39a98 4 4 0 15360000 15357640 PO-B /u/informix/space/archdb
4aa39c68 5 5 0 1000000 999947 PO-B /u/informix/space/tempdbs1
4aa39e38 6 6 0 1000000 999947 PO-B /u/informix/space/tempdbs2
4aa7e018 7 7 0 1000000 49947 PO-B /u/informix/space/phylogdbs
4aa7e1e8 8 8 0 1000000 499947 PO-B /u/informix/space/llogdbs
8 active, 32766 maximum
NOTE: The values in the "size" and "free" columns for DBspace chunks are
displayed in terms of "pgsize" of the DBspace to which they belong.
Expanded chunk capacity mode: always
Warning message in log file:
12:29:54 Warning: Invalid (non-existent/blobspace/disabled) dbspace listed
in DBSPACETEMP: 'tempdbs1'
12:29:54 Warning: Invalid (non-existent/blobspace/disabled) dbspace listed
in DBSPACETEMP: 'tempdbs2'
The database is not dead.
Just a REALLY WACKY thought, is it possible that there is a second engine
online on this machine that does not have those two temp dbspaces and that
it is configured to use the same message log file?
Are all of the oninit processes accounted for in the onstat -g glo list for
this instance?
Are there two (or more) sets of shared memory segments set up on the system
"ipcs -a|fgrep 0x56"?
(each instance will have a set of shared memory segments whose keys each
start with 0x5652 + SERVERNUM - so if SERVERNUM is 1 the segments will be
keyed 0x56534801, 0x56534802, etc. and if the SERVERNUM is 16 they would be
0x56624801, 0x56624802, etc.
Art
On Tue, Mar 17, 2009 at 12:44 PM, LAKSHMI DEVI PALANISSAMY <
lakshmidevip@hcl.in> wrote:
> DBSPACETEMP are tempdbs1 and tempdbs2
>
> Output of onstat -d:
> IBM Informix Dynamic Server Version 11.50.UC3 -- On-Line -- Up 00:00:33 --> 1091824 Kbytes
>
> Dbspaces
> address number flags fchunk nchunks pgsize flags owner name
> 4a09a808 1 0x40001 1 1 2048 N B informix rootdbs
> 4a09ac18 2 0x40001 2 1 2048 N B informix dbspace1
> 4a09ad78 3 0x40001 3 1 2048 N B informix traindb
> 4aa39018 4 0x40001 4 1 2048 N B informix archdb
> 4aa39178 5 0x42001 5 1 2048 N TB informix tempdbs1
> 4aa392d8 6 0x42001 6 1 2048 N TB informix tempdbs2
> 4aa39438 7 0x40001 7 1 2048 N B informix phylogdbs
> 4aa39598 8 0x40001 8 1 2048 N B informix llogdbs
> 8 active, 2047 maximum
>
> Chunks
> address chunk/dbs offset size free bpages flags pathname
> 4a09a968 1 1 0 1000000 993665 PO-B /u/informix/space/rootdbs
> 4aa396f8 2 2 0 5120000 5117636 PO-B /u/informix/space/dbspace1
> 4aa398c8 3 3 0 512000 509640 PO-B /u/informix/space/traindb
> 4aa39a98 4 4 0 15360000 15357640 PO-B /u/informix/space/archdb
> 4aa39c68 5 5 0 1000000 999947 PO-B /u/informix/space/tempdbs1
> 4aa39e38 6 6 0 1000000 999947 PO-B /u/informix/space/tempdbs2
> 4aa7e018 7 7 0 1000000 49947 PO-B /u/informix/space/phylogdbs
> 4aa7e1e8 8 8 0 1000000 499947 PO-B /u/informix/space/llogdbs
> 8 active, 32766 maximum
>
> NOTE: The values in the "size" and "free" columns for DBspace chunks are
>
> displayed in terms of "pgsize" of the DBspace to which they belong.
>
> Expanded chunk capacity mode: always
>
> Warning message in log file:
> 12:29:54 Warning: Invalid (non-existent/blobspace/disabled) dbspace listed
>
> in DBSPACETEMP: 'tempdbs1'
> 12:29:54 Warning: Invalid (non-existent/blobspace/disabled) dbspace listed
>
> in DBSPACETEMP: 'tempdbs2'
>
> The database is not dead.
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
--0016361e8134c0c42b04655383b5
We dont have any second online engine available.
I am using a script to call all these commands in a sequence i mentioned
already!
The error i reported was not consitently reproducable at all the times.
Hence i had a second thought and i tried the following:
1. After restore, ontape -r, i waited in a while loop until i see an instance
of oninit.
2. onmode -m
3. onmode -kyv
4. oninit -v
As i suspected, the problem was onmode -m. A simple sleep of 20 seconds
between 2nd and 3rd command actually resolved the problem.
I cleared the database and started in a clear slate. Its working fine without
any warnings or errors.
But instead of sleep, can i check for any instances and issue onmode -kyv.
What is the state of the database after issuing onstat -m.
Thanks,
Lakshmi
The state after an onmode -m should be the same as after an oninit -v fully
online once fast recovery completes. One thing you can do between the
onmode -m and the onmode -ky is to run onstat - in a loop and test thereturn code ($?). Onstat returns the current state flag for the engine
which will be '2' during recovery and '5' once the engine is online. So
simply loop until the return value is correct:
...
onmode -m
state=1
while [[ state -ne 5 ]]; do
sleep 5
onstat - >/dev/null 2>&1
state=$?
done
onmode -ky
oninit -v
Art
On Tue, Mar 17, 2009 at 1:29 PM, LAKSHMI DEVI PALANISSAMY <
lakshmidevip@hcl.in> wrote:
> We dont have any second online engine available.
>
> I am using a script to call all these commands in a sequence i mentioned
> already!
> The error i reported was not consitently reproducable at all the times.
> Hence i had a second thought and i tried the following:
> 1. After restore, ontape -r, i waited in a while loop until i see an
> instance
> of oninit.
> 2. onmode -m
> 3. onmode -kyv
> 4. oninit -v
>
> As i suspected, the problem was onmode -m. A simple sleep of 20 seconds
> between 2nd and 3rd command actually resolved the problem.
> I cleared the database and started in a clear slate. Its working fine
> without
> any warnings or errors.
>
> But instead of sleep, can i check for any instances and issue onmode -kyv.
>
> What is the state of the database after issuing onstat -m.
>
> Thanks,
> Lakshmi
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
--0016364ee1302c2fd80465545e0b
Hi Art!!
I tried with the return value of "onstat -d"
It is working fine with no issues.
Thanks for your timely help!! :-)
Thanks,
Lakshmi
Related threads
- IDS 10 table-level restore
- Informix Development Webinar December 11, 2007
- ontape -p/r with changed ROOTPATH
- Migrate from HP PA-RISC to HP ITANIUM by ontape