RE: Dumb DBA mistakes

Isn't it funny how many come from the backup and restore area. But how about this one:- A customer called us to help them recover a table that was deleted in error. They weren't using the same hardware company as we use. They hadn't upgraded despite many warnings, and their reserve machine didn't have enough disk space to recover the full system. Of course they only wanted to recover one table ... And they expected me to work miracles - again. And then there's the company who proudly told the auditors that they could recover their accounts system to a different machine in less than 48 hours. After more than one week of trying to recover AIX they then hit the hurdle of the 39 steps of imported restore. And then there is the company who maintain a spreadsheet of what software they have running on which platforms. One does wonder who their DBAs are :-) Once again I am sworn to secrecy as to who the companies are. I think the secret of this is - always test ALL scenarios you might need for data recovery - and keep testing them. Regards Malcolm -----Original Message----- From: informix-list-bounces@iiug.org [mailto:informix-list-bounces@iiug.org] On Behalf Of Martin Fuerderer Sent: 23 May 2006 08:22 To: informix-list@iiug.org; informix-list-bounces@iiug.org Subject: Re: Dumb DBA mistakes There goes one from the backup and restore area: DBAs worked out a very elaborate backup plan for their IDS instance using ontape and nightly archiving of different levels. Level 0 on Sundays, level 1 on Mondays, Wednesdays and Fridays, level 2 on Tuesdays and Thursdays. Tapes were meticuously changed every day, right on schedule for the nightly archiving. In addition they kept all tapes for 3 weeks and after that 1 weekly tape (level 0) for 3 months and again after that 1 monthly tape (level 0) forever. All the tapes nicely and orderly stored for easy and precise access should one be needed. As the day came (after years of strictly adhering to the above scheme), they pulled out the archive tape for restore, but it didn't work. As it can be possible that something's wrong with one tape, they tried the next tape, also not restorable. Over time they found out that none of the archives on the tapes was restorable. As it turned out, all the tapes were empty. The nightly ontape commands to do the archiving were never executed. Something went wrong with the cron setup, but nobody noticed. Nobody ever checked an archive for restorability. Nobody noticed that there never was a message about an archive started / completed in the IDS message log. They had many unused tapes in their storage. In the end they had to settle on some dbexport data that was 1.5 years old but still existed to restore their system ... Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management Visit one of the upcoming Infobahn events in Germany to get up-to-date information on Informix products: - June 20, 2006 at the IBM Forum Muenchen - June 21, 2006 at the IBM Forum Frankfurt - June 22, 2006 at the IBM Forum Berlin For more please see: http://www.ibm.com/de/events/infobahn/ _______________________________________________ Informix-list mailing list Informix-list@iiug.org http://www.iiug.org/mailman/listinfo/informix-list