Recovery failure - production system down
Posted in 2000
Hi All,
Any help gratefully accepted.
We are running INFORMIX-OnLine Version 7.23.UC11 (I know we
need to upgrade - my manager is resistant) on AIX 4.3.2 on
RS/6000
Disk failed (and not mirrored - 9.1 Gb SSA), earlier in the week.
Replaced the disk, rebuilt logical volumes.
Restore failing after successful physical recovery, and successful
roll forwrd of logical logs. On answering 'N' to any more logical logs:
11:46:54 Logical Recovery ABORTED. There are still open transactions; recovery cannot complete.
Restore done in quiescent mode, and as far as I am aware, nothing
else is on the system.
Currently awaiting Informix Tech Support feedback. Any ideas ???
Any thoughs gratefully accepted.
Dale Spence.
Restored dbspaces effected:
$ ontape -r -D dbsbsp1 dbsbsp3 ret dbsacr01
Please mount tape 1 on /dev/rmt0 and press Return to continue ...
Archive Tape Information
Tape type: Archive Backup Tape
Online version: INFORMIX-OnLine Version 7.23.UC11
Archive date: Sun Jul 30 08:04:25 2000
User id: informix
Terminal id: ?
Archive level: 0
Tape device: /dev/rmt0
Tape blocksize (in k): 512
Tape size (in k): 68000000
Tape number in series: 1
Continue restore? (y/n)y
Spaces to restore:1 [dbsacr01 ]
2 [ret ]
3 [dbsbsp1 ]
4 [dbsbsp3 ]
Please mount tape 2 on /dev/rmt0 and press Return to continue ...
Please mount tape 3 on /dev/rmt0 and press Return to continue ...
Restore a level 1 archive (y/n) n
Do you want to restore log tapes? (y/n)y
Roll forward should start with log number 303045
Please mount tape 1 on /dev/splive78.log and press Return to
continue ...
Do you want to restore another log tape? (y/n)y
...
Please mount tape 42 on /dev/splive78.log and press Return to
continue ...
Do you want to restore another log tape? (y/n)y
Please mount tape 43 on /dev/splive78.log and press Return to
continue ...
Do you want to restore another log tape? (y/n)y
Please mount tape 44 on /dev/splive78.log and press Return to
continue ...
Do you want to restore another log tape? (y/n)n
Logical restore failed - There are still open transactions; recovery
cannot comp
lete.
Program over.
At the time of the failed restore:
----------------------------------------------
$ onstat -u
INFORMIX-OnLine Version 7.23.UC11 -- Quiescent -- Up 14:10:50 -
- 526032 Kbytes
Userthreads
address flags sessid user tty wait tout locks nreads
nwrites
5003a014 ---P--D 1 informix - 0 0 0 206 1134
5003a454 ---P--F 0 informix - 0 0 0 0
312705
5003a894 ---P--F 0 informix - 0 0 0 0
272728
5003acd4 ---P--F 0 informix - 0 0 0 0
255310
5003b114 ---P--F 0 informix - 0 0 0 0
298661
5003b554 ---P--F 0 informix - 0 0 0 0
298261
5003b994 ---P--F 0 informix - 0 0 0 0
287941
5003bdd4 ---P--F 0 informix - 0 0 0 0
295527
5003c214 ---P--F 0 informix - 0 0 0 0
294301
5003c654 ---P--F 0 informix - 0 0 0 0
293804
5003ca94 ---P--F 0 informix - 0 0 0 0
316840
5003ced4 ---P--F 0 informix - 0 0 0 0
237510
5003d314 ---P--F 0 informix - 0 0 0 0
269655
5003d754 ---P--B 11 informix - 0 0 0 0 0
5003db94 ---P--D 13 informix - 0 0 0 0 0
15 active, 128 total, 28 maximum concurrent
onstat -l
INFORMIX-OnLine Version 7.23.UC11 -- Quiescent -- Up 14:15:25 -
- 526032 Kbytes
Physical Logging
Buffer bufused bufsize numpages numwrits pages/io
P-2 0 16 892 115 7.76
phybegin physize phypos phyused %used
2900035 10000 2824 0 0.00
Logical Logging
Buffer bufused bufsize numrecs numpages numwrits recs/pages
pages/io
L-1 0 16 54 54 54 1.0 1.0
address number flags uniqid begin size used %used
31ca0064 1 U-B---- 303481 d00035 2000 2000
100.00
31ca0080 2 U-B---- 303482 d00805 2000 2000
100.00
31ca009c 3 U-B---- 303483 d00fd5 2000 2000 100.00
31ca00b8 4 U-B---- 303484 d017a5 4000 4000
100.00
31ca00d4 5 U-B---- 303485 2506d63 2500 2500
100.00
31ca00f0 6 U-B---- 303486 d02745 8000 8000 100.00
31ca010c 7 U-B---- 303487 d04685 8000 8000
100.00
31ca0128 8 U-B---- 303488 2500003 8000 8000
100.00
31ca0144 9 U-B---- 303489 2501f43 8000 8000
100.00
31ca0160 10 U-B---- 303490 2503e83 8000 8000
100.00
31ca017c 11 U---C-L 303491 3f00003 8000 5661
70.76
31ca0198 12 U-B---- 303462 3f01f43 8000 8000
100.00
31ca01b4 13 U-B---- 303463 3f03e83 8000 8000
100.00
31ca01d0 14 U-B---- 303464 3f05dc3 8000 8000
100.00
31ca01ec 15 U-B---- 303465 3f07d03 8000 8000
100.00
31ca0208 16 U-B---- 303466 3f09c43 8000 8000
100.00
31ca0224 17 U-B---- 303467 3f0bb83 8000 8000
100.00
31ca0240 18 U-B---- 303468 d065c5 4000 4000
100.00
31ca025c 19 U-B---- 303469 2505dc3 4000 4000
100.00
31ca0278 20 U-B---- 303470 3f0dac3 4000 4000
100.00
31ca0294 21 U-B---- 303471 2b00003 8000 8000
100.00
31ca02b0 22 U-B---- 303472 2b01f43 8000 8000
100.00
31ca02cc 23 U-B---- 303473 2b03e83 8000 8000
100.00
31ca02e8 24 U-B---- 303474 2b05dc3 8000 8000
100.00
31ca0304 25 U-B---- 303475 2b07d03 8000 8000
100.00
31ca0320 26 U-B---- 303476 2b09c43 8000 8000
100.00
31ca033c 27 U-B---- 303477 2b0bb83 8000 8000
100.00
31ca0358 28 U-B---- 303478 2b0dac3 8000 8000
100.00
31ca0374 29 U-B---- 303479 2b0fa03 8000 8000
100.00
31ca0390 30 U-B---- 303480 2b11943 8000 8000
100.00
$ onmode -g ath
INFORMIX-OnLine Version 7.23.UC11 -- Quiescent -- Up 14:12:16 -
- 526032 Kbytes
Threads:
tid tcb rstcb prty status vp-class name
2 500afd18 0 2 sleeping(Forever) 8lio lio vp 0
3 500cb074 0 2 sleeping(Forever) 9pio pio vp 0
4 500cb3ec 0 2 sleeping(Forever) 10aio aio vp 0
5 500cb764 0 2 sleeping(Forever) 11msc msc vp
0
6 500cbc0c 0 2 sleeping(Forever) 12aio aio vp 1
7 500e710c