RE: Recovering data from a raw device....

Actually re-creating the tables is the first step, so the situation hasn't necessarily been compounded. Basically what you can do, is trick the engine into thinking that the data is still there. At this point I would try simply dd'ing the data back to the disks, and then try doing a select, you might get lucky. If the tables were re-created in the same order, and the extent sizes are the same, then there's a good chance that at least some of your tables will be recoverable. Obviously don't do this on a production system, and don't do it if there is any other data on the disks that you don't want to lose. If you don't have any luck, you can try re-creating the tables in a different order (until you get it right), and with different extent sizes, and then dd again. I've done something similar with a mirrored disk as a backup copy with good results. It's analogous to moving around .dbs files in standard engine. As long as the schema is the same, the engine doesn't know it's looking at a different file. Good luck. -----Original Message----- From: Chris Hall [mailto:Chris.Hall@OrbisUK.com] Sent: Monday, December 11, 2000 1:49 PM Posted To: informix Conversation: Recovering data from a raw device.... Subject: Recovering data from a raw device.... I find myself in an odd situation: Someone (no, not me) has just dropped all the tables from a database which, for reasons unknown, hasn't been backed-up for some time (i.e. months). To compound the error, new definitions of those tables were then added, for example: drop table table1;... drop table tableN; create table table1 ( <columns> ); create table table2 ( <columns> ); I'll find the reason for the backup failures and arrange for the appropriate penalty to be exacted, but in the meantime, all I have is a "dd"'d image of the raw partition where the data resided. I can see with "od" that there is some meaningful data left, and it seems to be in fairly large table-related contiguous chunks (the tables were originally created with sensible extent sizes). Are there any tools, official or otherwise, which will allow me to extract anything meaningful from this mess? I'd be grateful for any straws to clutch at... Chris -- ************************************************************** Chris Hall Email: Chris.Hall@OrbisUK.com Orbis Tel: +44 208 742 1600 http://www.OrbisUK.com Fax: +44 208 742 2649

Scott Black wrote in message <913g4s$j57$1@news.xmission.com>... > >Actually re-creating the tables is the first step, so the situation hasn't >necessarily been compounded. Basically what you can do, is trick the engine >into thinking that the data is still there. At this point I would try >simply dd'ing the data back to the disks, and then try doing a select, you >might get lucky. If the tables were re-created in the same order, and the >extent sizes are the same, then there's a good chance that at least some of >your tables will be recoverable. > >Obviously don't do this on a production system, and don't do it if there is >any other data on the disks that you don't want to lose. > >If you don't have any luck, you can try re-creating the tables in a >different order (until you get it right), and with different extent sizes, >and then dd again. I've done something similar with a mirrored disk as a >backup copy with good results. > >It's analogous to moving around .dbs files in standard engine. As long as >the schema is the same, the engine doesn't know it's looking at a different >file. Excepting that the plan will most likely fail unless the table partitions land in EXACTLY the right place, because the definitions of the locations of the extents and table spaces is stored in another place, and unlike with UNIX files (in standard engine) you can't spoof that. The Online Administrators Guide describes the structure fairly well, and if you are comfortable with Perl you may get some good results. PS - why are there dd images available but no archives? dd'ing Online spaces is a really pointless thing to do. This is not Oracle. I don't care what arguments people may come up with to justify the use of dd. Why not just do things directly and simply with the correct tools? If people claim that dd is a decent alternative backup method - bullshit! you have to have the engine offline or at least quiescent without even administrative fiddling going on. An archive can be done live on a changing system. The benefits of using online archives are WAY ahead of any silly dd tricks. Please don't argue with me, or I'll stick my fingers in my ear and go "Blah Blah Blah" No doubt you are feeling exactly the same way right now, Chris! Good luck.

In article <3a356546$1@news.iprimus.com.au>, "Andrew Hamm" <ahamm@sanderson.net.au> wrote: > Scott Black wrote in message <913g4s$j57$1@news.xmission.com>... > > > >Actually re-creating the tables is the first step, so the situation hasn't > >necessarily been compounded. Basically what you can do, is trick the > engine > >into thinking that the data is still there. At this point I would try > >simply dd'ing the data back to the disks, and then try doing a select, you > >might get lucky. If the tables were re-created in the same order, and the > >extent sizes are the same, then there's a good chance that at least some of > >your tables will be recoverable. > > > >Obviously don't do this on a production system, and don't do it if there is > >any other data on the disks that you don't want to lose. > > > >If you don't have any luck, you can try re-creating the tables in a > >different order (until you get it right), and with different extent sizes, > >and then dd again. I've done something similar with a mirrored disk as a > >backup copy with good results. > > > >It's analogous to moving around .dbs files in standard engine. As long as > >the schema is the same, the engine doesn't know it's looking at a different > >file. > > Excepting that the plan will most likely fail unless the table partitions > land in EXACTLY the right place, because the definitions of the locations of > the extents and table spaces is stored in another place, and unlike with > UNIX files (in standard engine) you can't spoof that. > > The Online Administrators Guide describes the structure fairly well, and if > you are comfortable with Perl you may get some good results. > > PS - why are there dd images available but no archives? dd'ing Online spaces > is a really pointless thing to do. This is not Oracle. I don't care what > arguments people may come up with to justify the use of dd. Why not just do > things directly and simply with the correct tools? If people claim that dd > is a decent alternative backup method - bullshit! you have to have the > engine offline or at least quiescent without even administrative fiddling > going on. An archive can be done live on a changing system. The benefits of > using online archives are WAY ahead of any silly dd tricks. Please don't > argue with me, or I'll stick my fingers in my ear and go "Blah Blah Blah" Well, I will argue with you. Sometimes it is neccessary to use tools other than those provided by informix. 1. If there are issues with the backup such that your backup does not complete successfully on a regular basis. So yes in this case dd(actually not really dd, but pretty close to the same thing) is more than just an alternative. 2. I do not consider the use of dd a 'trick'. I have never heard of a dd restore failing with out user error or a media failure. Essentially because of the simplicity of what you are doing there is not much to go wrong. On the other hand I have heard of situations in the past where an ontape backup ended up not being restorable. Don't get me wrong, I am a strong advocate of ontape/onbar(I can't claim to advocate onarchive), but suitability depends on the requirements of the system, and dd is sometimes suitable. Hope you didn't plug your ears, Will Sent via Deja.com http://www.deja.com/ Before you buy.

William Rice wrote in message <915bo3$ce3$1@nnrp1.deja.com>... > >Well, I will argue with you. Sometimes it is neccessary to use tools >other than those provided by informix. > >1. If there are issues with the backup such that your backup does not >complete successfully on a regular basis. So yes in this case >dd(actually not really dd, but pretty close to the same thing) is more >than just an alternative. > >2. I do not consider the use of dd a 'trick'. I have never heard of a >dd restore failing with out user error or a media failure. Essentially >because of the simplicity of what you are doing there is not much to go >wrong. On the other hand I have heard of situations in the past where >an ontape backup ended up not being restorable. > >Don't get me wrong, I am a strong advocate of ontape/onbar(I can't claim >to advocate onarchive), but suitability depends on the requirements of >the system, and dd is sometimes suitable. > >Hope you didn't plug your ears, >Will No, 'cos you snuck up on me and said it all while I was finishing a level in Pokemon. All fingers were attached to the controller and none were available for ear-filling duties. All I can say is, I remember some of our sites having problems with archives etc, and with a bit of work we have never had a problem since. These problems were ones I inherited 5 years ago when I landed here. Diving into the books and implementing (under protest at first) the rules suggested by Informix solved the problem. The main thrust of the work was to get sufficient space into the physical and logical logs so that all things may happily function during the archives. I vaguely remember a parameter being specifically important here, but it really escapes me now 'cos it has been absorbed into the general setup rules. I assume you are able to take the engine offline for the duration of your dd-style archive? If not, how do you address the consistency problem should you have to do a dd restore?

<SNIP> > > All I can say is, I remember some of our sites having problems with archives > etc, and with a bit of work we have never had a problem since. These > problems were ones I inherited 5 years ago when I landed here. Diving into > the books and implementing (under protest at first) the rules suggested by > Informix solved the problem. The main thrust of the work was to get > sufficient space into the physical and logical logs so that all things may > happily function during the archives. I vaguely remember a parameter being > specifically important here, but it really escapes me now 'cos it has been > absorbed into the general setup rules. 1. Not all backup failures are due to incorrect configuration... 2. Even if they are due to incorrect configuration, using dd, or some other such utility is a good interim solution until those issues are resolved. > > I assume you are able to take the engine offline for the duration of your > dd-style archive? If not, how do you address the consistency problem should > you have to do a dd restore? > > Yes, the system would have to be unavailable to the users during the backup. In some systems that is acceptable. Like I said before I am a strong advocate for ontape/onbar, but if someone tells me they due a dd backup once a week, just in case, if their system allows it, all the better for them. To be honest, the particular situation I was thinking of, I inherited the system after Informix and the previous DBA had finally gotten a backup to complete successfully. The initial problem was a bug which caused the backup to hang forever. I am not sure what the resolution was, seeing it was solved before I started. Until such time as we get a system to test our restore on, I am not comfortable with the backup. That is my policy on any backup, it is useless until you test the restore, preferrably multiple times. Especially, if problems were encountered in the past. We have tested our raw partition backups with our system. They worked. Nothing better than a backup which you have tested. Will Sent via Deja.com http://www.deja.com/

William Rice wrote in message <917v4u$hek$1@nnrp1.deja.com>... > >1. Not all backup failures are due to incorrect configuration... > >2. Even if they are due to incorrect configuration, using dd, or some >other such utility is a good interim solution until those issues are >resolved. We are definitely drifting into deeply philosophical positions here. 1) Can't argue - engine bugs can be a real pisser. We've been fairly fortunate because we've been able to either test fairly extensively on inhouse machines, or on "new" customer machines (exploit the pre-live testing to also prove the engines), although we have definitely had to pull a few engine versions in the past despite all that. None that I can recall were due to backup issues. Sadly, we've had to test a few backups, and they've all worked every time. 2) Seriously, I'd rather put the effort into getting (what I consider) "proper" practices into place, testing new engines prior to deployment, giving the flick to dud software, solving failures of essential services. This is where the philosophical divergence sets in. But you are clearly comfortable with dd as a "backup" backup and I'm comfortable not to put any effort into using it. "You say tomato and I say tomato" - hmmm - that line doesn't really work when written down ;-) >To be honest, the particular situation I was thinking of, I inherited >the system after Informix and the previous DBA had finally gotten a >backup to complete successfully. The initial problem was a bug which >caused the backup to hang forever. I am not sure what the resolution >was, seeing it was solved before I started. Until such time as we get a >system to test our restore on, I am not comfortable with the backup. >That is my policy on any backup, it is useless until you test the >restore, preferrably multiple times. Especially, if problems were >encountered in the past. We have tested our raw partition backups with >our system. They worked. Nothing better than a backup which you have >tested. Once bitten, twice shy. I've got my own peculiar rituals and obsessive/compulsive behaviours based on shit that has happened in the past... We pretend to be logical, scientific and precise, but every so often good ol' human nature wins over in difficult situations. Cheers, and may all your backups be both error free and unused.