Re: Can Disk mirroring/db snapshots replace the usage of taking database backups?

"Art S. Kagel (Oninit)" <art@oninit.com> wrote in message news:mailman.909.1207759535.20610.informix-list@iiug.org... > jpierrot@chubb.com wrote: >> >> Hi Gang, >> I am evaluating a software out there called FalconStor and the software >> company promises to provide a solution that would make backing up the >> informix database or any other dbms using native commands such as onbar, >> rman, etc unnecessary. They said that their software takes a snapshot of >> the disk at regular interval from 5 minutes to one hour, depending on the >> frequency you want to set it. Then If you want to restore to any point it >> time , you just have to point your dbms to that snapshot and you are >> current as the time that snapshot was taken. Data within instances can >> also be refreshed from one environment to another using the same method. >> > > This will not work consistently and so not at all. IDS keeps much > information about the state of the database server and its disk structures > in memory only flushing it to disk at checkpoint time. At best restarting > an IDS instance pointed to such a copy would require a long recovery > period. At worst it will not work at all. Not as an automated scheduled > copy. There is a way to take archived by copying the disks directly in > IDS 10.00 and later. It is called External Backup and Restore. Read the > manuals for the requirements, the point is that the engine has to be > involved and certain onmode commands have to be executed before the copy > is begun and after it is completed. Interesting point, Art. My experience is that you are correct, and that onmode -c block could be used with the Falconstar backups (I have noexperience with Falconstar actually but assume they are similar to other vendors' snapshot technology) and external restores to achieve what J Pierrot wants. Conceptually, though, i've never been sure why the onmode -c block is needed. As you say, "much information about the state of the database server and its disk structures in memory only flushing it to disk at checkpoint time". But, to say Informix cannot necessarily recover from an unblocked disk copy is surely exactly the same as saying that Informix cannot recover from any unexpected system crash?! And, as we know, it does.

Neil Truby wrote: > "Art S. Kagel (Oninit)" <art@oninit.com> wrote in message > news:mailman.909.1207759535.20610.informix-list@iiug.org... > >> jpierrot@chubb.com wrote: >> >>> Hi Gang, >>> I am evaluating a software out there called FalconStor and the software >>> company promises to provide a solution that would make backing up the >>> informix database or any other dbms using native commands such as onbar, >>> rman, etc unnecessary. They said that their software takes a snapshot of >>> the disk at regular interval from 5 minutes to one hour, depending on the >>> frequency you want to set it. Then If you want to restore to any point it >>> time , you just have to point your dbms to that snapshot and you are >>> current as the time that snapshot was taken. Data within instances can >>> also be refreshed from one environment to another using the same method. >>> >>> >> This will not work consistently and so not at all. IDS keeps much >> information about the state of the database server and its disk structures >> in memory only flushing it to disk at checkpoint time. At best restarting >> an IDS instance pointed to such a copy would require a long recovery >> period. At worst it will not work at all. Not as an automated scheduled >> copy. There is a way to take archived by copying the disks directly in >> IDS 10.00 and later. It is called External Backup and Restore. Read the >> manuals for the requirements, the point is that the engine has to be >> involved and certain onmode commands have to be executed before the copy >> is begun and after it is completed. >> > > Interesting point, Art. My experience is that you are correct, and that > onmode -c block could be used with the Falconstar backups (I have no> experience with Falconstar actually but assume they are similar to other > vendors' snapshot technology) and external restores to achieve what J > Pierrot wants. > > Conceptually, though, i've never been sure why the onmode -c block is > needed. As you say, "much information about the state of the database > server and its disk structures in memory only flushing it to disk at > checkpoint time". But, to say Informix cannot necessarily recover from an > unblocked disk copy is surely exactly the same as saying that Informix > cannot recover from any unexpected system crash?! And, as we know, it does. > I agree, IDS's fast recovery is plenty smart and will usually cope with anything. And yet, occasionally, a fast recovery fails because some commit record that spanned a disk sector didn't all get on disk and the engine is confused requiring a call to IBM and a Down System Engineer to dial in and truncate the logical logs in order to get things back online. I just prefer the known quantity of a nice safe ontape archive in my pocket. Art S. Kagel Oninit

Art S. Kagel (Oninit) wrote: > Neil Truby wrote: >> "Art S. Kagel (Oninit)" <art@oninit.com> wrote in message >> news:mailman.909.1207759535.20610.informix-list@iiug.org... >> >>> jpierrot@chubb.com wrote: >>> >>>> Hi Gang, >>>> I am evaluating a software out there called FalconStor and the >>>> software company promises to provide a solution that would make >>>> backing up the informix database or any other dbms using native >>>> commands such as onbar, rman, etc unnecessary. They said that their >>>> software takes a snapshot of the disk at regular interval from 5 >>>> minutes to one hour, depending on the frequency you want to set it. >>>> Then If you want to restore to any point it time , you just have to >>>> point your dbms to that snapshot and you are current as the time >>>> that snapshot was taken. Data within instances can also be refreshed >>>> from one environment to another using the same method. >>>> >>>> >>> This will not work consistently and so not at all. IDS keeps much >>> information about the state of the database server and its disk >>> structures in memory only flushing it to disk at checkpoint time. At >>> best restarting an IDS instance pointed to such a copy would require >>> a long recovery period. At worst it will not work at all. Not as an >>> automated scheduled copy. There is a way to take archived by copying >>> the disks directly in IDS 10.00 and later. It is called External >>> Backup and Restore. Read the manuals for the requirements, the point >>> is that the engine has to be involved and certain onmode commands >>> have to be executed before the copy is begun and after it is completed. >>> >> >> Interesting point, Art. My experience is that you are correct, and >> that onmode -c block could be used with the Falconstar backups (I have >> no experience with Falconstar actually but assume they are similar to >> other vendors' snapshot technology) and external restores to achieve >> what J Pierrot wants. >> >> Conceptually, though, i've never been sure why the onmode -c block is >> needed. As you say, "much information about the state of the database >> server and its disk structures in memory only flushing it to disk at >> checkpoint time". But, to say Informix cannot necessarily recover >> from an unblocked disk copy is surely exactly the same as saying that >> Informix cannot recover from any unexpected system crash?! And, as we >> know, it does. > > > I agree, IDS's fast recovery is plenty smart and will usually cope with > anything. And yet, occasionally, a fast recovery fails because some > commit record that spanned a disk sector didn't all get on disk and the > engine is confused requiring a call to IBM and a Down System Engineer to > dial in and truncate the logical logs in order to get things back online. > I just prefer the known quantity of a nice safe ontape archive in my > pocket. > > Art S. Kagel > Oninit > > If a fast recovery needs human intervention (and nothing wrong happened to the disks) than you're facing a bug... This should be true to any relational database system with logging (or redo? logs). Database must know exactly what was written and what wasn't... Things like disk controllers cache can confuse this..., but in any case is a bug... Regards -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently...

Hello Neil, the engine has to be prevented from writing to disk when someone is createing a copy > server and its disk structures in memory only flushing it to disk at > checkpoint time". But, to say Informix cannot necessarily recover from an > unblocked disk copy is surely exactly the same as saying that Informix > cannot recover from any unexpected system crash?! And, as we know, it does. How about lru cleaning.... you are looking at a moving target here... if one has copied the physical log using say dd and is also copying it's datadbs and lru cleaning kicks in, the first thing it will do is check if a page needs to be written to the physical log and if so it does (hmmmm you have already copyd that..... so you miss the before image. ) then it will write the page to it's spot on disk... hmmm after the engine has written that you copy the chunk...(so you have the new page..) that will for sure create problems and a useless backup. An unexpected system crash does not nuke the way the engine writes pages to disk.... it does control how it writes stuff and in what order. Superboer. way fast=http://www.clipjes.nl/clip/nederlands/n/normaal_- _oerend_hard.html

"Fernando Nunes" <spam@onlinedomus.net> wrote in message news:ftjj02$apt$1@aioe.org... > If a fast recovery needs human intervention (and nothing wrong happened to > the disks) than you're facing a bug... This should be true to any > relational database system with logging (or redo? logs). > > Database must know exactly what was written and what wasn't... Things like > disk controllers cache can confuse this..., but in any case is a bug... I agree. So, it begs the question: "What is onmode -c block for?" (in the context of "external" backups).

"Superboer" <superboer7@t-online.de> wrote in message news:214c06b0-fc94-4ebd-a2b3-4a679c9df666@y21g2000hsf.googlegroups.com... > Hello Neil, > > the engine has to be prevented from writing to disk when someone is > createing a copy > >> server and its disk structures in memory only flushing it to disk at >> checkpoint time". But, to say Informix cannot necessarily recover from >> an >> unblocked disk copy is surely exactly the same as saying that Informix >> cannot recover from any unexpected system crash?! And, as we know, it >> does. > > How about lru cleaning.... you are looking at a moving target here... > > if one has copied the physical log using say dd and is also copying > it's > datadbs and lru cleaning kicks in, the first thing it will do is check > if a page needs to be written to > the physical log and if so it does (hmmmm you have already copyd > that..... so you miss the before image. ) then it will write > the page to it's spot on disk... hmmm after the engine has written > that you copy the chunk...(so you have the new page..) > that will for sure create problems and a useless backup. Rightm, but all these snapshot-based technologies use "consistency group" functionality so what you describe above doesn't apply. I still don't understand - referring to the implication in Art's original reply - why a database block/unblock is necessary for an external backup.

> Rightm, but all these snapshot-based technologies use "consistency group" > functionality so what you describe above doesn't apply. > I still don't understand - referring to the implication in Art's original > reply - why a database block/unblock is necessary for an external backup. well i still would not trust "consistency group". i rather go for an onbar/ontape then trust another party tool. when this has a bug or goes wrong who are you gonna talk to/ who is gonna fix it.??!! if you have massive corruption, then this is absolutely no fun to repair it. when you really want the above to work i suggest to ask IBM/Informix to certify it. Also one thing i forgot is that the sys reserved pages are updated using a onmode -c block so the engine knows what logs to rollforward from. (check onstat -g arc/ oncheck -pr) Superboer. way fast=http://www.clipjes.nl/clip/nederlands/n/normaal_- _oerend_hard.html

"Superboer" <superboer7@t-online.de> wrote in message news:fffc5c22-82a8-4f19-ab32-f090f4c66179@u36g2000prf.googlegroups.com... >> Rightm, but all these snapshot-based technologies use "consistency group" >> functionality so what you describe above doesn't apply. >> I still don't understand - referring to the implication in Art's original >> reply - why a database block/unblock is necessary for an external backup. > > > well i still would not trust "consistency group". i rather go for an > onbar/ontape then trust another party > tool. > > when this has a bug or goes wrong who are you gonna talk to/ who is > gonna fix it.??!! > if you have massive corruption, then this is absolutely no fun to > repair it. That's fine and dandy, but be clear that what you are saying is functionally identical to saying that IDS Fast recovery cannot be relied upon either.

Hi, if I remember correctly, this topic has been discussed in great length some time ago in this forum/list (albeit with a different subject line). So please dig in the archive to get answers. Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management IBM Deutschland Entwicklung GmbH Chairman of the Supervisory Board: Martin Jetter Board of Management: Herbert Kircher Corporate Seat: Boeblingen, Germany Reg.-Gericht: Amtsgericht Stuttgart, HRB 243294 informix-list-bounces@iiug.org wrote on 10.04.2008 12:31:10: > "Superboer" <superboer7@t-online.de> wrote in message > news:214c06b0-fc94-4ebd-a2b3-4a679c9df666@y21g2000hsf.googlegroups.com... > > Hello Neil, > > > > the engine has to be prevented from writing to disk when someone is > > createing a copy > > > >> server and its disk structures in memory only flushing it to disk at > >> checkpoint time". But, to say Informix cannot necessarily recover from > >> an > >> unblocked disk copy is surely exactly the same as saying that Informix > >> cannot recover from any unexpected system crash?! And, as we know, it > >> does. > > > > How about lru cleaning.... you are looking at a moving target here... > > > > if one has copied the physical log using say dd and is also copying > > it's > > datadbs and lru cleaning kicks in, the first thing it will do is check > > if a page needs to be written to > > the physical log and if so it does (hmmmm you have already copyd > > that..... so you miss the before image. ) then it will write > > the page to it's spot on disk... hmmm after the engine has written > > that you copy the chunk...(so you have the new page..) > > that will for sure create problems and a useless backup. > > Rightm, but all these snapshot-based technologies use "consistency group" > functionality so what you describe above doesn't apply. > I still don't understand - referring to the implication in Art's original > reply - why a database block/unblock is necessary for an external backup. > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list

Hi, "onmode -c block" does more than just a (normal) checkpoint. One important thing is that it writes to the dbspaces the information about that same checkpoint. That way the restore can later verify that the dbspaces (externally restored) are of the same checkpoint (or not). This information is not written to the dbspaces with normal checkpoints (without block). (For those it is only kept in root reserved pages.) This is the main reason, why e.g. for external restore you can't use a copy of the dbspaces, with the copy being done after regular shutdown of IDS (i.e. with checkpoint done). It is not the same. For an external restore you could mix copies of different dbspaces from different times together. With the current implementation IDS would have a hard time figuring out each dbspace's "reference time". I agree that many things could be implemented differently. But I doubt this would be possible without incurring "cost" elsewhere, be it performance, space or convenience. Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management IBM Deutschland Entwicklung GmbH Chairman of the Supervisory Board: Martin Jetter Board of Management: Herbert Kircher Corporate Seat: Boeblingen, Germany Reg.-Gericht: Amtsgericht Stuttgart, HRB 243294 informix-list-bounces@iiug.org wrote on 10.04.2008 12:28:09: > "Fernando Nunes" <spam@onlinedomus.net> wrote in message > news:ftjj02$apt$1@aioe.org... > > > If a fast recovery needs human intervention (and nothing wrong happened to > > the disks) than you're facing a bug... This should be true to any > > relational database system with logging (or redo? logs). > > > > Database must know exactly what was written and what wasn't... Things like > > disk controllers cache can confuse this..., but in any case is a bug... > > I agree. > So, it begs the question: "What is onmode -c block for?" (in the context of > "external" backups). > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list