Linux onbar restore fails (IDS9.4)
Posted in 2003
Hello everybody,
I'm evaluating IDS 9.40.UC1E1 on Linux, specifically I was interested in
seeing how ONBAR works on this version (for backup/recovery).
I have a problem, I can backup, but a simple restore test fails.
Maybe somebody can help (or has suggestions); it's not very urgent, since
this is just a test.
1) Linux Environment
--------------------
First, my idea was to use IDS 9.40 on Slackware. I have used 9.30 on
Slackware 8.1 and it worked fine.
However, I get a segmentation fault:
oninit -ivInitializing Dictionary Cache and SPL Routine Cache...succeeded
Bringing up ADM VP...succeeded
<segmentation fault>
Does anyone know what ADM VP is ?
Anyway, because this seemed like a GLIBC problem, I tried Slackware 7.1
(glibc 2.1.3) , Slackware 8.0 and 8.1 (glibc 2.2.5) and Slackware 9.0 (glibc
2.3.1) but none works out of the box (contrary to IDS 9.30).
So I installed a RedHat 7.2 with glibc 2.2.4 and kernel 2.4.7.
Linux localhost.localdomain 2.4.7-10 #1 Thu Sep 6 17:27:27 EDT 2001 i686 unknown
-rwxr-xr-x 1 root root 1282588 Sep 4 2001 /lib/libc-2.2.4.so
That worked. If I do an 'oninit -iv' after the ADM VP step, it says:
Creating VP classes...succeeded
and everything continues to work, and the server goes online.
Next, I did an upgrade of the kernel on Linux RH 7.2.
I upgraded from modutils-2.4.6-4 to modutils-2.4.14-3; I then did a
rpm -U kernel-2.4.9-31.i686.rpm
After rebooting the system, I tried oninit -iv again, and it works fine.
2) Backup Environment
---------------------
Good; now that I have a fully supported RedHat 7.2 with a working IDS 9.40,
I configure Legato NMI 2.1 (Networker module for Informix) with NetWorker 6.1.3
After running onbar, I get:
awk: cmd. line:2: fatal: file `/opt/informix/etc/' is a directory
awk: cmd. line:2: fatal: file `/opt/informix/etc/' is a directory
Apparently there is a bug in the onbar script:
You have to add:
ONCONFIG=onconfig
because it assumes that ONCONFIG is set. Otherwise you get the AWK error.
Then I do a backup (onbar -b -w -L 0) and all is fine; backup works, the
simple database is stored in Legato (as I can verify there).
Next, the critical test comes (can I do a recover?)
So I shutdown IDS 9.40 (onmode -ky) and I start a cold restore:
[root@localhost dbs]# onbar -r -w
SENDER IS NULL NO MAIL WILL BE SENT
SENDER IS NULL NO MAIL WILL BE SENT
However my IDS 9.40 system is still not restored.
onstat - shows that IDS is not online.
I repeat the onbar -r -w recovery using BAR_DEBUG level 5 and here is the
output of the BAR_DEBUG.LOG file:
2003-05-28 10:34:56 2752 2750 bar_log_close_restore: input commit_flag 1
2003-05-28 10:34:56 2752 2750 bar_log_close_restore: output errtxt Archive API error: not yet open
2003-05-28 10:34:56 2752 2750 bar_log_close_restore: return 130 (0x82)
2003-05-28 10:34:56 2752 2750 do_log_restore: return 130 (0x82)
2003-05-28 10:35:01 2752 2750 bar_is_drsec: enter
2003-05-28 10:35:01 2752 2750 bar_is_drsec: return 0 (0x00)
2003-05-28 10:35:01 2752 2750 return from bar_exec 130
2003-05-28 10:35:01 2752 2750 bar_unloadXBSA: enter
2003-05-28 10:35:01 2752 2750 bar_unloadXBSA: return 0 (0x00)
2003-05-28 10:35:01 2752 2750 bar_main: return 130 (0x82)
Does anyone know what the 'Archive API : not yet open' can indicate ?
I have the impression that the cold restore worked fine (I can also do
a onbar -r -p rootdbs without problems), but the problem is after the
recovery of the logical log.
The recovery of the logical log also worked, but then later I get this
"Archive API errro".
In the BAR_ACT.LOG file I get:
2003-05-28 10:41:59 3138 3136 /opt/informix/bin/onbar_d -r -w
2003-05-28 10:42:00 3138 3136 Successfully connected to Storage Manager.
2003-05-28 10:42:30 3138 3136 Begin salvage for log 3.
2003-05-28 10:42:31 3138 3136 Completed salvage of logical log 3.
2003-05-28 10:42:31 3138 3136 Successfully connected to Storage Manager.
2003-05-28 10:43:02 3138 3136 Begin cold level 0 restore rootdbs (Storage Manager copy ID: 1054109958 1054109959).
2003-05-28 10:43:25 3138 3136 Completed cold level 0 restore rootdbs.
2003-05-28 10:43:27 3138 3136 Completed whole system restore.
2003-05-28 10:43:30 3138 3136 Successfully connected to Storage Manager.
2003-05-28 10:43:34 3138 3136 Begin restore logical log 3 (Storage Manager copy ID: 1054111320 1054111321).
2003-05-28 10:43:42 3138 3136 Completed restore logical log 3.
2003-05-28 10:43:56 3138 3136 Unable to close the storage space restore: buc_fe.c : Archive API processing failed at line 935 for msgtype.
2003-05-28 10:43:56 3138 3136 Unable to close the storage space restore: Archive API error: not yet open.
2003-05-28 10:44:01 3138 3136 /opt/informix/bin/onbar_d complete, returning 130 (0x82)
Error 130 is 'database error' according to the documentation.
In the /nsr/applogs/xbsa.messages there's nothing (NSR_DEBUG_LEVEL 8) that
I can see.
Anyone know what:
2003-05-28 10:43:56 3138 3136 Unable to close the storage space restore: buc_fe.c : Archive API processing failed at line 935 for msgtype.
can indicate ? How can I recover my test database ? (oninit -r seems to
go blocked, CPKT required).
If I simply type "oninit" I get the following in ONLINE.LOG:
10:30:32 Physical Recovery Started at Page (1:845).
10:30:32 Physical Recovery Complete: 0 Pages Examined, 0 Pages Restored.
10:30:32 Logical Recovery Started.
10:30:32 10 recovery worker threads will be started.
10:30:32 Assert Failed: Dynamic Server must abort
10:30:32 Informix Dynamic Server Version 9.40.UC1E1
10:30:32 Who: Session(8, informix@localhost.localdomain, 0, 0x10d87758)
Thread(18, fast_rec, 10d57858, 1)
File: rslog.c Line: 3396
10:30:32 Results: Dynamic Server must abort
10:30:32 Action: Reinitialize shared memory
10:30:32 stack trace for pid 2532 written to /tmp/af.3fa73a8
10:30:32 See Also: /tmp/af.3fa73a8, shmem.3fa73a8.0
10:30:35 rslog.c, line 3396, thread 18, proc id 2532, Dynamic Server must abort.
10:30:35 PANIC: Attempting to bring system down