Re: Ontape restore to another server

>From: "Keith Simmons" <smiley73@googlemail.com> >To: "Gary Quiring" <gquiring@gmail.com> >CC: informix-list@iiug.org [SNIP] >On 28 Dec 2006 07:57:44 -0800, Gary Quiring <gquiring@gmail.com> wrote: > > Sebastian, Norma J. wrote: > > > Don't think that's different than other DB products. > > > > > > Use the right tool (dbexport/import like Keith said), and it'll work > > > fine. > > My current backup is in ontape format. The only place to get that data > > is on that tape. I don't have another Sun box to restore the data to. > > Looking forward I don't see using dbexport as a practical backup > > solution for a 150gig DB. Other DB products may be alike but that does > > not excuse the issue. I think it's silly they can't make backup data > > compatible between hardware platforms. [SNIP] >Gary > >You were not asking for an archive/backup solution, only data >transfer. Horses for courses. Ontape is designed to archive data and >then restore it to the same or similar server. The two servers you >have probably have different page sizes (2K for Sol, 4K for Linux) >possibly different endedness (big or little), different procesors >(Sparc and Intel). Text files wold probably ftp between them OK but >ontape stores the data in binary page images (for speed of archive and >restore). >I would have thought cpio or tar wold not work between these servers >either, and they would only be transfering 'data'. >Do you have enough space on your Solaris box to create another >instance and then restore to this (using chunk renaming) just to >extract the table you need. >ou could investigate table level restores as well, although I can't >remember off the top of my head whether this is available at v9.4. > >Keith Uhm.... cpio and tar will work when transfering ascii files across different hardware. Binaries only across similar hardware. Going back many moons, I used to recieve mag tapes from clients and would have to unload the data and load it to a database. ... But back to the point... I believe Keith is correct regarding how Ontape works. Meaning that the binary image of the data page. Having said that... Note: THIS IS NOT INFORMIX SPECIFIC.... YMMV... In *theory*, you could write a utility that could convert the data from one format to another. I don't believe this would be trivial since you're not just merging/splitting pages but you have to go row by row and reset references and pointers. Each page of the database table has some header information that will most likely not only be platform specific but also version specific. Then on each row, you not only have the data, but you may also have pointers to other spaces like blobs and or VARCHAR data that didn't fit on the row. So if the page format changes, then these pointers will have to reflect the changes. Not really trivial. Also the issue of word size and "endian" would come in to play. But in *theory* you could do this. It would either take a non-disclosure with IBM or some time doing the necessary reverse engineering, or both. (Ok, knowing the specifics of the layout of a page in each version would be *very* important, so yeah. It would require an NDA ...) The downside is the following: 1) You'll need a lot of disk space to do the conversion. 2) Not much value unless you're doing a special disaster recovery or forensic computing. Also another twist... data encryption of the backup.... But hey! What do I know? ;-) _________________________________________________________________ Your Hotmail address already works to sign into Windows Live Messenger! Get it now http://clk.atdmt.com/MSN/go/msnnkwme0020000001msn/direct/01/?href=http://get.live.com/messenger/overview

> > > solution for a 150gig DB. Other DB products may be alike but that does > > > not excuse the issue. I think it's silly they can't make backup data > > > compatible between hardware platforms. As all the others have said anint on solaris is different then one on linux. and it is well doced that this is the case!!!! for your problem: i would suggest that you have a look at archecker. V10 allows table level restores documented, i guess V9.4 has it undoced?? dono, need to look at the manual, try Tech Support, they should be able to help you out. Superboer. Hmmm someone said cpio can be transfered to .... have you tryed a cpio file created on sco to extract on say sun solaris??? you may find a suprise there ...... Ian Michael Gumby schreef: > >From: "Keith Simmons" <smiley73@googlemail.com> > >To: "Gary Quiring" <gquiring@gmail.com> > >CC: informix-list@iiug.org > [SNIP] > >On 28 Dec 2006 07:57:44 -0800, Gary Quiring <gquiring@gmail.com> wrote: > > > Sebastian, Norma J. wrote: > > > > Don't think that's different than other DB products. > > > > > > > > Use the right tool (dbexport/import like Keith said), and it'll work > > > > fine. > > > My current backup is in ontape format. The only place to get that data > > > is on that tape. I don't have another Sun box to restore the data to. > > > Looking forward I don't see using dbexport as a practical backup > > > solution for a 150gig DB. Other DB products may be alike but that does > > > not excuse the issue. I think it's silly they can't make backup data > > > compatible between hardware platforms. > [SNIP] > >Gary > > > >You were not asking for an archive/backup solution, only data > >transfer. Horses for courses. Ontape is designed to archive data and > >then restore it to the same or similar server. The two servers you > >have probably have different page sizes (2K for Sol, 4K for Linux) > >possibly different endedness (big or little), different procesors > >(Sparc and Intel). Text files wold probably ftp between them OK but > >ontape stores the data in binary page images (for speed of archive and > >restore). > >I would have thought cpio or tar wold not work between these servers > >either, and they would only be transfering 'data'. > >Do you have enough space on your Solaris box to create another > >instance and then restore to this (using chunk renaming) just to > >extract the table you need. > >ou could investigate table level restores as well, although I can't > >remember off the top of my head whether this is available at v9.4. > > > >Keith > > Uhm.... > > cpio and tar will work when transfering ascii files across different > hardware. Binaries only across similar hardware. Going back many moons, I > used to recieve mag tapes from clients and would have to unload the data and > load it to a database. ... > > But back to the point... > > I believe Keith is correct regarding how Ontape works. Meaning that the > binary image of the data page. > > Having said that... > > Note: THIS IS NOT INFORMIX SPECIFIC.... YMMV... > > In *theory*, you could write a utility that could convert the data from one > format to another. > I don't believe this would be trivial since you're not just > merging/splitting pages but you have to go row by row and reset references > and pointers. > > Each page of the database table has some header information that will most > likely not only be platform specific but also version specific. Then on each > row, you not only have the data, but you may also have pointers to other > spaces like blobs and or VARCHAR data that didn't fit on the row. > > So if the page format changes, then these pointers will have to reflect the > changes. > Not really trivial. > > Also the issue of word size and "endian" would come in to play. > > But in *theory* you could do this. It would either take a non-disclosure > with IBM or some time doing the necessary reverse engineering, or both. > (Ok, knowing the specifics of the layout of a page in each version would be > *very* important, so yeah. It would require an NDA ...) > > The downside is the following: > 1) You'll need a lot of disk space to do the conversion. > 2) Not much value unless you're doing a special disaster recovery or > forensic computing. > > Also another twist... data encryption of the backup.... > > But hey! What do I know? ;-) > > _________________________________________________________________ > Your Hotmail address already works to sign into Windows Live Messenger! Get > it now > http://clk.atdmt.com/MSN/go/msnnkwme0020000001msn/direct/01/?href=http://get.live.com/messenger/overview

>From: "Superboer" <superboer7@t-online.de> > > > > solution for a 150gig DB. Other DB products may be alike but that >does > > > > not excuse the issue. I think it's silly they can't make backup data > > > > compatible between hardware platforms. > > >As all the others have said anint on solaris is different then one on >linux. >and it is well doced that this is the case!!!! > Really? You mean that there's more to the issue being big endian vs little endian? Wow. Must have missed that in Madison's example: ("Integers are not the same on intel chips as they are on the solaris sparc chip. It has nothing to do with the OS, but rather the format of integers used by the chip. For the number 67305985, Intel would use 0x01020304 while the sparc chip would use 0x04030201.") Seems to me, *thats* the only difference. Sure, there's the issue of word size, but I think everything these days is 64 bit. Although you *could* apply that to the filter as well. *This* issue is trival to solve, and, if you re-read my post, I could have sworn I mentioned this issue as well. ;-) The non-trivial portion deals with knowing the page layout of header information and data. Also you'll need to know something about the data layout itself. (There are some issues which are non-trivial and would require an NDA from IBM.) This will change from version to version of Informix, so you need to know which version of the database the "tape" came from. The other issue is what other data is on the backup and some of the basic formatting issues. Seriously, this isn't rocket science, however there are some things that can make this a very difficult task. An example... How does Informix handle reference pointers to data that can't fit on a row? Is this consistent from version to version? There's more, but hey! What do I know? ;-) -G _________________________________________________________________ Get live scores and news about your team: Add the Live.com Football Page www.live.com/?addtemplate=football&icid=T001MSN30A0701

Ian Michael Gumby wrote: > > > >> From: "Superboer" <superboer7@t-online.de> > >> > > > solution for a 150gig DB. Other DB products may be alike but >> that does >> > > > not excuse the issue. I think it's silly they can't make backup >> data >> > > > compatible between hardware platforms. >> >> >> As all the others have said anint on solaris is different then one on >> linux. >> and it is well doced that this is the case!!!! >> > > Really? > You mean that there's more to the issue being big endian vs little endian? > Wow. Must have missed that in Madison's example: > ("Integers are not the same on intel chips as they are on the solaris > sparc chip. It has nothing to do with the OS, but rather the format of > integers used by the chip. For the number 67305985, Intel would use > 0x01020304 while the sparc chip would use 0x04030201.") > > Seems to me, *thats* the only difference. Sure, there's the issue of > word size, but I think everything these days is 64 bit. Although you > *could* apply that to the filter as well. The endian issue is by far the biggest reason that you couldn't restore a Solaris backup onto a intel/Linux machine. That is unless the solaris backup is from an intel solaris machine. The page header is fairly trivial and could be done with a simple filter program. But you would still have problems with all of the flags, extent lists, partition pages, reserved pages, etc. > > *This* issue is trival to solve, and, if you re-read my post, I could > have sworn I mentioned this issue as well. ;-) > > The non-trivial portion deals with knowing the page layout of header > information and data. > Also you'll need to know something about the data layout itself. (There > are some issues which are non-trivial and would require an NDA from IBM.) > > This will change from version to version of Informix, so you need to > know which version of the database the "tape" came from. > > The other issue is what other data is on the backup and some of the > basic formatting issues. > > Seriously, this isn't rocket science, however there are some things that > can make this a very difficult task. > > An example... > How does Informix handle reference pointers to data that can't fit on a > row? Is this consistent from version to version? > > There's more, but hey! > > What do I know? ;-) > > -G > > _________________________________________________________________ > Get live scores and news about your team: Add the Live.com Football Page > www.live.com/?addtemplate=football&icid=T001MSN30A0701 >

>Ian Michael Gumby wrote: >> >> >> >>>From: "Superboer" <superboer7@t-online.de> >> >>> > > > solution for a 150gig DB. Other DB products may be alike but that >>>does >>> > > > not excuse the issue. I think it's silly they can't make backup >>>data >>> > > > compatible between hardware platforms. >>> >>> >>>As all the others have said anint on solaris is different then one on >>>linux. >>>and it is well doced that this is the case!!!! >>> >> >>Really? >>You mean that there's more to the issue being big endian vs little endian? >>Wow. Must have missed that in Madison's example: >>("Integers are not the same on intel chips as they are on the solaris >>sparc chip. It has nothing to do with the OS, but rather the format of >>integers used by the chip. For the number 67305985, Intel would use >>0x01020304 while the sparc chip would use 0x04030201.") >> >>Seems to me, *thats* the only difference. Sure, there's the issue of word >>size, but I think everything these days is 64 bit. Although you *could* >>apply that to the filter as well. > >The endian issue is by far the biggest reason that you couldn't restore a >Solaris backup onto a intel/Linux machine. That is unless the solaris >backup is from an intel solaris machine. > Ok Madison... You're still working for HAL, right? Then if you're only blocked by the endian issue, you should be able to whip this thing out pronto. HINT: Set a boolean flag to true if you have to flip the word when you're converting from one format to the other. If boolean is true, then call a c function to "flip the bits". Then write out the returned value to your output stream. Remember its not just "integers" but all words would be flipped. >The page header is fairly trivial and could be done with a simple filter >program. But you would still have problems with all of the flags, extent >lists, partition pages, reserved pages, etc. > > Well *you* can say the page headers would be trivial. ;-) I haven't seen the source code nor have I tried to reverse engineer this. ;-) Flags would be trivial since you would know the two formats so its a simple translation. But yes, its all the address pointers. This wouldn't be "trivial" but again its not rocket science. You'd have to keep a queue page of address points and then resolve them. There are a couple of possible designs, the simplest being would require some memory, and a fair amount of disk space. Plan on 2.5 times the size of the initial tape size. You could improve upon this with some logic too... Again, in theory, its not that difficult. Figure a small 2-3 man team, 6 months time or less. Write it in C and ESQL/C. Note: Write the body of the code work in C and then the DBPageWrite routine in ESQL/C. (You could even add connection pooling and some parallelism after you get it working.) Development cost ? Mythical man month = 160 hour month. Rate? $125 an hour USD. So if you can get $360,000 USD from management, you could build this tool. The bottom line, such a beast is doable. However, I'm not sure that it would get approved by IM's management team. Its a neat tool, however there are a limited set of use cases to justify its existence. Having said that, I can think of a couple of things that *may* justify it. ;-) But hey! What do I know? ;-) _________________________________________________________________ Get live scores and news about your team: Add the Live.com Football Page www.live.com/?addtemplate=football&icid=T001MSN30A0701

Ian Michael Gumby wrote: > > But hey! What do I know? ;-) > I don't know what you know, but I _am_ tired of you asking. What little credibility you build up in your posts is washed away every fucking time you use this tagline, it is really annoying. Get some new material.

Ian Michael Gumby wrote: <SNIP> > Remember its not just "integers" but all words would be flipped. > <SNIP> Careful. No ALL WORDS will be flipped. Intel, unlike some old little-endian chips (DEC K10/K20 chips most notably) does not flip the contents of words within a character array. Also, since DECIMAL and it's derived types in IDS (DATETIME, INTERVAL, etc.) are not native types they are also not byte flipped. So before you willy-nilly flip every word of data on the tape you have to process the table schema's for each table. So, this is NOT trivial. Now, could someone update archecker to be able to read cross platform tapes from a different -endian platform? Sure, it only has to deal with one table at a time and 95% of the work's already done. However, making ontape or onbar do that is a HUGELY different story. Art S. Kagel

Ian Michael Gumby wrote: <SNIP> > Remember its not just "integers" but all words would be flipped. > <SNIP> Careful. No ALL WORDS will be flipped. Intel, unlike some old little-endian chips (DEC K10/K20 chips most notably) does not flip the contents of words within a character array. Also, since DECIMAL and it's derived types in IDS (DATETIME, INTERVAL, etc.) are not native types they are also not byte flipped. So before you willy-nilly flip every word of data on the tape you have to process the table schema's for each table. So, this is NOT trivial. Now, could someone update archecker to be able to read cross platform tapes from a different -endian platform? Sure, it only has to deal with one table at a time and 95% of the work's already done. However, making ontape or onbar do that is a HUGELY different story. Art S. Kagel

> > But hey! What do I know? ;-) > > > >I don't know what you know, but I _am_ tired of you asking. What little >credibility you build up in your posts is washed away every fucking time >you use this tagline, it is really annoying. Get some new material. Timmy, is that you? ;-) Did you wake up on the wrong side of the bed this morning? The tag line goes way back to the days of Finnochio and Dexmier. Its a polite way of putting things in to perspective. As gumby, I'm just an "anonymous" voice on the internet that may or may not be full of shite. Its a reminder that we should all be egoless programmers and that we shouldn't take ourselves too seriously. There's more to it of course.... But hey! What do I know? -G HAPPY NEW YEAR! _________________________________________________________________ Get live scores and news about your team: Add the Live.com Football Page www.live.com/?addtemplate=football&icid=T001MSN30A0701