Re: dbexport(working) / dbimport (not working) using named pipe

Hi All, dbimport 2gb issue again; As expected dbimport is failing once it reaches 2GB limit. Instead of asking the next tape it's failing with *** load table *** Read tape failed *** Import data is corrupted! 0 - Unknown error message 0. It seems my client doesn't have a support agreement with Informix. Looking at the release notes I've found below defects still remaining open even latest version of IDS 7.31 family. Can somebody (Informix Tech. Support guys hello) verify whether this is a bug for IDS 7.31.UC5 or not ? PRODUCT: DBATOOLS 54311 DBIMPORT WITH BLOCKSIZE OF 32 AND TAPESIZE OF 2097151 CAUSES READ TAPE FAILED MESSAGE AT END OF FIRST TAPE Date:2002/07/03 Version: 7.31.UD5 CUSTOMER-REPORTED BUGS STILL OPEN WITH THE RELEASE OF 7.31.UD5 146032 DBATOOLS - DBIMPORT DBIMPORT FAILS WITH READ TAPE FAILED, IMPORT DATA CORRUPTED,UNKNOWN ERROR MESSAGE 0, AND TAPE SIZE SET TO 2097151 AND BLKSIZE OF 16 132316 DBATOOLS - DBIMPORT DBIMPORT FAILS WITH READ TAPE FAILED WHEN ATTEMPTING TO READ A TAPE WITH A FIXED RECORD SIZE LESS THAN 1024 BYTES Any idea, input is appreciated. Thanks in Advance. "Alkin Tezuysal" <alkin.tezuysal@gmail.com> wrote in message news:IY-dneYy2NPhfLXeRVn-3w@comcast.com... > Thanks for both the answers. > > 8 kb seems to be working for now. I'll be trying on a large db (about > 35gb) very soon > > and see what happens. > > Thanks, > > Alkin T. > > > > Might want to try using a block size of 8 (kbytes) for the (pipe) > stream.... I traced ontape, and this is the size it required. Perhaps the > others share the same concept... > > > > > > > > > Martin Fuerderer > > <MARTINFU@de.ibm. To: "Alkin Tezuysal" <alkin.tezuysal@gmail.com> > > com> cc: informix-list@iiug.org, owner-informix-list@iiug.org > > Sent by: Subject: Re: dbexport / dbimport using named pipe > > owner-informix-li > > st@iiug.org > > > > 09/13/2005 03:58 > > AM > > > > > > > > > > Hi, > > the problem is, that there is a difference between a tape device and a > pipe. dbimport is made for working with tape devices. This also works well > for files in a file system without too much difference and dbimport > supports files as well (albeit in your version only up to 2 GB, that's > correct). > > A pipe (named or not) has different characteristics than other devices. > Particularly, a pipe can have a state where there is no data in the pipe > because the writer to the pipe is slower than the reader. > > With a pipe this is a transient but not a fatal problem. The reader only > has to wait until new data arrives from the writer. Once that has > happened, the reader can continue reading (and actually getting) data from > the pipe. Therefore the reader from a pipe has to repeatedly try to read > from the pipe when there is no data ... until new data arrives or a fatal > error happens (e.g. pipe has been destroyed). > > Contrary, with a tape/file device a read that does not get any data means > that there is no more data. Any retry is pointless. If it happens before > all expected data has been read, then this is a fatal condition. > > And this is how dbimport handles it. > > dbimport does not have special provisions for handling a pipe, therefore > you are seeing this problem. Of course you can try to minimize the > probability of the problem by giving the gzip (the writer to the pipe) a > head start, e.g. put a sleep of several seconds inbetween the gzip and > dbimport commands. However, there will never be a guarantee that it will > always work. It may also depend on how big the pipe can become - which may > be a configurable parameter in the OS ... ? > > On some platforms, the "tar" system utility provides parameter "-B" > > exactly to solve this sort of problem. It will make tar to repeatedly read > from a pipe if it is empty, or to repeatedly try to write to the pipe if > it is currently full. To my knowledge no such thing has been implemented > for dbexport and dbimport. > > Regards, > > Martin > > -- > > Martin Fuerderer > > IBM Informix Development Munich, Germany Information Management > > owner-informix-list@iiug.org wrote on 13.09.2005 01:29:50: > >> Hi, > >> I've constructed below after researching from the cdi archive. > >> Initially it worked for a small db and now it fails for larger. > >> > >> dbexport> >> > >> mknod /tmp/test p > >> gzip -9f </tmp/test >/tmp/test.exp & > >> dbexport test -t /tmp/test -f /tmp/test.sql -b 16 -s 12000000 -ss > >> > >> > >> dbimport> >> > >> mknod /tmp/test p > >> cd /tmp > >> gunzip -c > /tmp/test < /tmp/test.exp & dbimport test -c -t /tmp/test > >> -b 16 -s 12000000 -f /tmp/test.sql > >> > >> After creating the db and loading couple tables it fails with below > > message. > >> > >> *** load table *** > >> Read tape failed > >> > >> *** Import data is corrupted! > >> > >> 0 - Unknown error message 0. > >> > >> From the same gzip file If I were to > >> > >> gunzip -c > /tmp/test < /tmp/test.exp > >> > >> and > >> > >> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql > >> > >> import completes successfully. > >> > >> My goal is to create one export file and put it to tape while > >> tackling informix 2Gb file size limit on IDS 7.31 UC5 family. > >> > >> Any input is appreciated. > >> > >> Thanks, > >> > >> Alkin T. > >> > >> Informix DBA > >> > >> alkin.tezuysal@gmail.com > > sending to informix-list > > > > > > > > This e-mail is intended only for the addressee named above. > > As this e-mail may contain confidential or privileged information, if you > are not the named addressee, you are not authorized to retain, read, copy > or disseminate this message or any part of it. > > "Alkin Tezuysal" <alkin.tezuysal@gmail.com> wrote in message > news:mdudnW3v_4jqkrveRVn-vw@comcast.com... >> Hi, >> I've constructed below after researching from the cdi archive. >> Initially it worked for a small db and now it fails for larger. >> >> dbexport>> >> mknod /tmp/test p >> gzip -9f </tmp/test >/tmp/test.exp & >> dbexport test -t /tmp/test -f /tmp/test.sql -b 16 -s 12000000 -ss >> >> >> >> dbimport>> >> mknod /tmp/test p >> cd /tmp >> gunzip -c > /tmp/test < /tmp/test.exp & >> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql >> >> After creating the db and loading couple tables it fails with below >> message. >> >> *** load table *** >> Read tape failed >> >> *** Import data is corrupted! >> >> 0 - Unk