Re: Re: (Another) ontape to recover from a FIFO pipe

Hi Jacob, Here's an example that uses pipes and compression. Use at your own risk, and assume it is pseudocode and has not been tested. I wrote it quickly last week to help Martha (IIUG member) with the same sort of issue. Please let me know your thoughts and suggestions. I'm posting to the IIUG because of the many requests to do what you are wanting to do... #!/bin/sh -e #... #... Remember to setup your Informix environment variables #... FILEMB=100 # Mbytes maximum per file created RDIR=/local/db_tmp/Martha # location of compress--split files TLINK=/tmp/tapedev.link # onconfig: TAPEDEV /tmp/tapedev.link # onconfig: TAPEBLK 8 # streams # onconfig: TAPESIZE 2000000000 # big touch $TLINK # temporary for testing,,, if [ 1 -eq 1 ]; then #.....................Perform the Backup #... The database should be online before running :-) #... The backup can be run in the background :-) #... rm $TLINK # remove link mknod $RDIR/split.pipe p # make a pipe mknod $RDIR/backup.pipe p # make a pipe ln -s $RDIR/backup.pipe $TLINK # create link to the pipe chown -R informix:informix $RDIR # informix ownership to contents chmod 660 $RDIR/*.pipe # read/write permission to pipes compress -fv - < $RDIR/backup.pipe > $RDIR/split.pipe & split -b${FILEMB}m -a 5 - $RDIR/file_ < $RDIR/split.pipe & echo "\\n\\n" | ontape -s -L 0 # backup program rm $RDIR/*.pipe # cleanup pipes touch $TLINK # temporary for testing,,, fi if [ 1 -eq 0 ]; then #.....................Perform the Restore #... The database should be offline before running :-) #... The restore should be run in the foreground :-) #... rm $TLINK # remove link mknod $RDIR/restore.pipe p # make a pipe ln -s $RDIR/restore.pipe $TLINK # create link to the pipe chown -R informix:informix $RDIR # informix ownership to contents chmod 660 $RDIR/restore.pipe # read/write permissions to pipe ( # work-around for "rewind" cat `ls -1 $RDIR/file_[a-z]*` | uncompress -c > $RDIR/restore.pipe || \\ cat `ls -1 $RDIR/file_[a-z]*` | uncompress -c > $RDIR/restore.pipe )& echo "wait 5 seconds before replying to prompts" ontape -r # restore program rm $RDIR/*.pipe # cleanup pipesfi Kind Regards, -Tim "Art S. Kagel" <kagel@bloomberg.n To: informix-list@iiug.org et> cc: Sent by: Subject: Re: (Another) ontape to recover from a FIFO pipe owner-informix-lis t@iiug.org 10/13/2004 05:28 PM Please respond to kagel On Wed, 13 Oct 2004 16:49:24 -0400, Jacob Salomon wrote: You cannot recover from a pipe. Ontape opens the 'tape' file, reads the reserved pages, performs some calculations, closes the 'tape' file, and reopens expecting that the tape has rewound. Art S. Kagel > Informix IDS 7.23 (Go ahead and laugh - I agree with you) > > Hi Family. > > I have been researching ways to archive to a named pipe. I see from > researching this newsgroup that I am far from the dfirst person to try this. > By playing with the TAPEBLK size, I came up with 128K as an ideal buffer > size for backups - it backed up a 140GB system in under 2 hours. > > The Unix SA created a 200GB "large file" file system while I created a FIFO > and set TAPEDEV to reference it. I also set the TAPESIZE to a billion (K). > All set up. > > I backed up by starting a DD command to read from the FIFO and write to a > flat file. The I started the ontape -s -L 0. Like I said above, it was done > in under 2 hours using the 128K block size. > > Now how about recovery? Well, I started the ontape -r in one window. $ > ontape -r> Please mount tape 1 on /apps/informix/arc_pipe and press Return to continue > ... > > Before hitting enter, I ran this dd command in another window: $ dd > if=/inf_archive/archives/archive.L0 \\ > of=/apps/informix/arc_pipe bs=131072 > > That is, read back from the flat file I created in the backup (79GB) and > write it to the FIFO in the same block size with which it was created. > > On the ontape window, I hit <Return>. ontape displays the dbspaces, presents > the ontape -d output and asks: Continue restore? (y/n) y Of course Y silly! > > I answer "n" to "Do you want to back up the logs?". > > Now it starts to churn away for a few seconds. Then POW: > > Physical restore failed - restore reserved pages failed Program over. > > Bak to the dd side, that program is patiently trying to write to the FIFO. I > CTRL-C out of it and I get the message: 9+0 records in 8+0 records out > > Hey, if the reserved paged could not restore, the restore should have died > at the first "record" from dd - at 128K it certainly covers the reserved > pages and then some! But Noooo! It read 8 records of 128K each - a full meg > - before deciding it couldn't handle the reserved pages. > > I cannot possibly be the first fool to stumble into this problem! Has anyone > else ever had this problem and solved it? Is anyone out there successfully > archiving and recovering to/from flat files using a FIFO pipe file? > > Thanks. > +------------------------------------------------- Jacob Salomon -+ |When > you exit this vehicle, please be sure to lower your head and| |watch your > step. If you fail to do so, please lower your voice | |and watch your > language. | > +-----------------------------------------------------------------+ sending to informix-list