Logical Log Tape Validator

This is a multi-part message in MIME format. --------------2DB122E2544D Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit The following perl script was written after some very painful experiences with informix backup and recovery as follows: 1. Last level 0 5 days before but continuos logical log backup running. 2. Accidently dropped a BIG table and deleted it's exported contents 3. PANIC firing squad at dawn etc. - then brainwave: 4. Stop logical log backup 5. New perl script to look for a new table name in tape and note tape block. (it was extremely easy) 6. Second script to tape-tape copy the required number of blocks (ie without my drop) 7. New level 0 to be on the safe side. 8. Restored prev level 0 - put in copied logical tape and: 9. CRASHED - no tape header (does not ask you to try again) 10.Had to go public - firing squad prepared 11.Informix consulted - no useful help (try original tape) 11.Comparisons with other tapes showed the original tape had no header. 12.Patched copied tape with a (2048 bytes) header in first block 13.Restored level 0 - put in patched logical log tape and: 14.ERROR Tape contains wrong logical log - try another (dident crash this time) 15.Put in original tape (ie Informix's advice) and: 16.CRASHED - no tape header 17.Studied tape further and found that although tbtape said it had written logs 5111-5319 it only contained 5154-5319 and no header. We assume it must have rewound the tape with no reported errors or Online log entry. 18.Back to previous level 0 19.Load - Aim - Fire Conclusions: Apart from the obvious - level 1/2 backups etc. (actually not possible due to round the clock conversion of blob data which could not be stopped and the archive causes it to crash - Another informix gripe - it should be called INFORMIX SOMETIMES) Reports from tbtape and online log cannot be relied on. You must know that the tape you are using during a recovery is a logical log tape with a valid header. Otherwise the whole recovery will CRASH (this is unforgivable with modern!! software. Similarly Archive Create will CRASH if you accidently put in a write-protected tape.) I have turned one of the scripts into the log tape validator enclosed in the hope it may be of use to other informix lusers. This script was written using perl 5 and informix 5.06. I have no idea if it will work with perl4 and/or other informix versions. I am certain that if my original log tape had been OK the ruse of copying it up to a certain block would have worked and no one would have known. I supposes life is like that sometimes. If you dont have perl you realy must get it. It is FREE from http://www.perl.org - go for 5.004 now. Also get the DBI and Informix DBD modules. These allow direct embedded SQL ala ESQL/C and can even read blobs into variables. Please note that the above comments are my personal opinions and not those of my company. John Clutterbuck Never eat on an empty stomach. #!/usr/local/bin/perl # informix 5.x logical log tape validator # This software is supplied without any warrenty whatsover # If you find it useful use at your risk but dont blame me # J.Clutterbuck 11th June 1997 $usage="logtape_validate.pl [-v] ltapedev\\n"; #check for optional -v (verbose) argument if ( $ARGV[0] eq "-v" ) { $verbose=1; shift @ARGV; } else { $verbose=0; } if ( $#ARGV != 0 ) { die $usage; } $LOGTAPE=$ARGV[0]; #get environment variables $TBCONFIG=$ENV{'TBCONFIG'} || die "Cannot evaluate TBCONFIG environment variable\\n"; $INFORMIXDIR=$ENV{'INFORMIXDIR'} || die "Cannot evaluate INFORMIXDIR environment variable\\n"; #open TBCONFIG file open(TBCONFIG,"<$INFORMIXDIR/etc/$TBCONFIG") || die "cannot open $INFORMIXDIR/etc/$TBCONFIG for reading\\n"; #read and look for LOGSIZE & LTAPEBLK print "Getting logical log tape details from $INFORMIXDIR/etc/$TBCONFIG\\n"; while (<TBCONFIG>) { if ( /^LOGSIZE\\s+([0-9]+)\\s+/ ) { $LOGSIZE=$1; } elsif ( /^LTAPEBLK\\s+([0-9]+)\\s+/ ) { $LTAPEBLK=$1; } } close(TBCONFIG); if ( ! $LOGSIZE || ! $LTAPEBLK ) { die "Cannot establish LOGSIZE or LTAPEBLK from $INFORMIXDIR/etc/$TBCONFIG\\n"; } #open the log file open(LOGTAPE,"<$LOGTAPE") || die "Cannot open $LOGTAPE\\n"; print "Reading tape: $LOGTAPE\\n"; #read each line $found=0; $num=0; $rec_bytes = $LTAPEBLK * 1024; #number of bytes per tape record $log_bytes = $LOGSIZE * 1024; #number bytes per log file $num_per_log = 1 + int(($log_bytes-1)/$rec_bytes); #number of records per log $got_err=0; $found_hdr=0; $curr_logno=0; $first_logno=0; $prev_logno=0; $recno=0; nextlog: while(1) { #read all the records per log file $found_log=0; foreach $b ( 1 .. $num_per_log ) { $numread=sysread(LOGTAPE,$logbuf,$rec_bytes); ++$recno; if ( $numread != $rec_bytes ) { if ( $found_hdr && $curr_logno ) { print "ERROR $numread BYTES READ INSTEAD OF $rec_bytes IN RECORD $recno - END OF TAPE?\\n" if $verbose; } else { print "ERROR $numread BYTES READ INSTEAD OF $rec_bytes IN RECORD $recno - NOT A LOG FILE TAPE\\n" if $verbose; $got_err=1; } last nextlog; } #look at header(s) if first record per log file if ( $b >= 1 ) { $hdr_offset=0; testhdr: while(1) { #unpack header from first 2048 bytes $header=substr($logbuf,$hdr_offset,2048); ($h1)=unpack "i4",$header; #look for tape header if ( $h1 == -1 ) { printf "TAPE HEADER (%d) record %4d offset %5d\\n",$h1,$recno,$hdr_offset if $verbose; if ( ++$found_hdr > 1 || $hdr_offset || $recno != 1) { print "*** ERROR - DUPLICATE OR NOT IN FIRST BLOCK ***\\n"; $got_err=1; last nextlog; } } elsif ( $h1 == -4 ) { ++$curr_logno; ($log_number)=unpack "i4",substr($header,16,4); $message=''; if ( $curr_logno == 1 && ! $found_hdr ) { $message.="*** ERROR - NO TAPE HEADER ***"; $got_err=1; } if ( ++$found_log > 1 ) { $message.="*** ERROR - LOG $found_log IN BLOCK ***"; $got_err=1; } #check for log sequence and remeber first/last if ( $prev_logno ) { if ( $log_number != $prev_logno+1 ) { $message.="*** ERROR - BAD LOG SEQUENCE $prev_logno - $curr_logno ***"; $got_err=1; } } elsif ( ! $first_logno ) { $first_logno=$log_number; } $prev_logno=$log_number; printf "Log Header (%d) record %4d offset %5d - LOGFILE: %6d %s\\n",$h1,$recno,$hdr_offset,$log_number,$message if $verbose; } elsif ( $h1 > -10 && $h1 < 0 ) { printf "???? Header (%d) record %4d offset %5d\\n",$h1,$recno,$hdr_offset if $verbose; } else { last testhdr; } #look at next bi