Re: SNAP & SAN

"Martin Fuerderer" <MARTINFU@de.ibm.com> wrote in message news:mailman.189.1155122544.20706.informix-list@iiug.org... > Hi, > > I try to clarify this a bit more ... > > After restoring the snapshot, you have an "image" of the data > exactly as it was while the server was blocked when taking > the snapshot. The command "onmode -c block" explicitly > writes a checkpoint so that all data will be on disk and the > disks will be physically consistent. Only after that will the > command "onmode -c block" return the prompt to the user. > > However, this physical consistency does not mean that things > are logically consistent, i.e. regarding transactions, etc. > > Because of this restoring the snapshot actually corresponds > to a physical only restore of the dbspaces. > > After a physical only restore, a logical restore should follow > to make things logically consistent. This is the logical restore phase. > The minimum work required to be done in general is to roll back > any open transactions. This needs to be done even if there are > no logs to restore. I've been thinking about this a bit more. I've never quite understood why the onmode -c block is necessaary, because even without it - and assuming that the SAN is clever enough to ensure that writes across the set of LUNs that make up the snapshot unit are self-consistent, which it surely will - the external recovery should be able to restore the database to its state of the last checkpoint. If this *wasn't* the case then fast recovery in any system, whether part of a SAN or not, wouldn't work. Without logical recovery, as I understand it, the fast recovery restores the database server back to its checkpoint state, rolls forward transactions from the logical log, then backs out incomplete ones at the time of thye snap, thereby restoring all completed transactions up to the point of the snap.

Neil Truby wrote: > Gesendet: 14 August 2006 21:28 > An: informix-list@iiug.org > Betreff: Re: SNAP & SAN > > "Martin Fuerderer" wrote: > > After restoring the snapshot, you have an "image" of the data > > exactly as it was while the server was blocked when taking > > the snapshot. The command "onmode -c block" explicitly > > writes a checkpoint so that all data will be on disk and the > > disks will be physically consistent. Only after that will the > > command "onmode -c block" return the prompt to the user. ... > > I've been thinking about this a bit more. I've never quite > understood why > the onmode -c block is necessaary, because even without it - > and assuming > that the SAN is clever enough to ensure that writes across > the set of LUNs > that make up the snapshot unit are self-consistent, which it > surely will - > the external recovery should be able to restore the database > to its state of > the last checkpoint. If this *wasn't* the case then fast > recovery in any > system, whether part of a SAN or not, wouldn't work. The checkpoint and block of server is necessary as IDS currently relies on a checkpoint for a start of logical recovery or rollback of open transactions. If we did not write a checkpoint before starting to take the snapshot there would be 'old' stale pages on disk, while the correct image was still in IDS' BUFFER. Also the IDS needs to be blocked while the snapshot is initiated because the snapshot needs to be taken after all pages were written to disk, but before users start modifying and writing more pages after the checkpoint. If you used an ordinary checkpoint it would be impossible to time this. It would be possible to request some sort of special checkpoints at configurable intervals that write a checkpoint and then trigger a configurable external program that initiates a snapshot. Only after a return of that external program the checkpoint would then finish and user activity could resume. Would classify as feature request for IDS next. Regards Tilman

"Model-Bosch, Tilman" <tilman.model-bosch@sap.com> wrote in message news:mailman.266.1155586222.20706.informix-list@iiug.org... Neil Truby wrote: > Gesendet: 14 August 2006 21:28 > An: informix-list@iiug.org > Betreff: Re: SNAP & SAN > > "Martin Fuerderer" wrote: > > After restoring the snapshot, you have an "image" of the data > > exactly as it was while the server was blocked when taking > > the snapshot. The command "onmode -c block" explicitly > > writes a checkpoint so that all data will be on disk and the > > disks will be physically consistent. Only after that will the > > command "onmode -c block" return the prompt to the user. ... > > I've been thinking about this a bit more. I've never quite > understood why > the onmode -c block is necessaary, because even without it - > and assuming > that the SAN is clever enough to ensure that writes across > the set of LUNs > that make up the snapshot unit are self-consistent, which it > surely will - > the external recovery should be able to restore the database > to its state of > the last checkpoint. If this *wasn't* the case then fast > recovery in any > system, whether part of a SAN or not, wouldn't work. >> The checkpoint and block of server is necessary as IDS currently relies on a checkpoint for a start of logical recovery or rollback of open transactions. If we did not write a checkpoint before starting to take the snapshot there would be 'old' stale pages on disk, while the correct image was still in IDS' BUFFER. I'm not sure I buy this argument. Think for a minute about the situation where, just at the point that we're about to the take the onmode -c block, we pull the plug on the live server. The primary disk on the SAN is in exactly the same state as the copy - it has old, stale pages on disk where the correct image was in the IDS buffer, until the plug was pulled. But we know for a fact that when we re-start the server, fast recovery will use the physical and logical logs on the primary disk to recover to a point of logical consistency just before the plug was pulled - such recovery is fundamental to Informix. But if the SAN copy is identical to the primary disk it follows that a Informix server started against it will *also* fast recover perfectly. So what purpose does the onmode -c block serve? >> Also the IDS needs to be blocked while the snapshot is initiated because the snapshot needs to be taken after all pages were written to disk, but before users start modifying and writing more pages after the checkpoint. If you used an ordinary checkpoint it would be impossible to time this. Why? The same argument as above applies. Fast recovery will cope with it on the main server on the primary disk, and since the SAN copy is by definition an identical copy of the primary, fast recovery can recover on the copy too.

Hi, the questions here are: - how long does the snapshot take ? - how often are checkpoints written ? - what other write activity happens on disks ? (e.g. foreground writes ... ?) If you start your snapshot (starting with the first disk), and a new checkpoint happens writing on that first disk after it was copied, but also on the other disks before they are copied, then you have a physical inconsistency. E.g. a blobspace BLOB is inserted. The BLOB itself goes onto disk A that was already copied, but the table (and the blob structure) are on disk B that still is to be backed up. Then you will later have a missing BLOB, because that BLOB in the blobspace on disk A didn't make it onto the snapshot. Whereas IDS cannot detect at the time that this snapshot is inconsistent - it would not even know that a snapshot was done. This is just one example, you can construct others ... I'm not sure how "clever" a SAN and its snapshot functionality are, but I certainly would not trust it to detect and correctly handle examples like that above. And certainly I would not rely on the assumption that "the snapshot is so fast that it is done before any writes can introduce inconsistencies". If you insist in trying your method (without blocking), then I advice you to run extensive "oncheck" commands after the restore. If it finds inconsistencies, then you simply do the whole snapshot exercise again ... that is if the system is still around for that. TIA, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management IBM Information On Demand Global Conference October 15-20, 2006, Anaheim, California see http://www.ibm.com/events/informationondemand informix-list-bounces@iiug.org wrote on 14.08.2006 23:03:44: > "Model-Bosch, Tilman" <tilman.model-bosch@sap.com> wrote in message > news:mailman.266.1155586222.20706.informix-list@iiug.org... > Neil Truby wrote: > > Gesendet: 14 August 2006 21:28 > > An: informix-list@iiug.org > > Betreff: Re: SNAP & SAN > > > > "Martin Fuerderer" wrote: > > > After restoring the snapshot, you have an "image" of the data > > > exactly as it was while the server was blocked when taking > > > the snapshot. The command "onmode -c block" explicitly > > > writes a checkpoint so that all data will be on disk and the > > > disks will be physically consistent. Only after that will the > > > command "onmode -c block" return the prompt to the user. > ... > > > > I've been thinking about this a bit more. I've never quite > > understood why > > the onmode -c block is necessaary, because even without it - > > and assuming > > that the SAN is clever enough to ensure that writes across > > the set of LUNs > > that make up the snapshot unit are self-consistent, which it > > surely will - > > the external recovery should be able to restore the database > > to its state of > > the last checkpoint. If this *wasn't* the case then fast > > recovery in any > > system, whether part of a SAN or not, wouldn't work. > > >> The checkpoint and block of server is necessary as IDS currently relies > on a checkpoint for a start of logical recovery or rollback of > open transactions. > If we did not write a checkpoint before starting to take the > snapshot there would be 'old' stale pages on disk, while the > correct image was still in IDS' BUFFER. > > I'm not sure I buy this argument. Think for a minute about the situation > where, just at the point that we're about to the take the onmode -c block, > we pull the plug on the live server. The primary disk on the SAN is in > exactly the same state as the copy - it has old, stale pages on disk where > the correct image was in the IDS buffer, until the plug was pulled. But we > know for a fact that when we re-start the server, fast recovery will use the > physical and logical logs on the primary disk to recover to a point of > logical consistency just before the plug was pulled - such recovery is > fundamental to Informix. But if the SAN copy is identical to the primary > disk it follows that a Informix server started against it will *also* fast > recover perfectly. > > So what purpose does the onmode -c block serve? > > >> Also the IDS needs to be blocked while the snapshot is initiated > because the snapshot needs to be taken after all pages were written > to disk, but before users start modifying and writing more pages > after the checkpoint. If you used an ordinary checkpoint it would > be impossible to time this. > > Why? The same argument as above applies. Fast recovery will cope with it > on the main server on the primary disk, and since the SAN copy is by > definition an identical copy of the primary, fast recovery can recover on > the copy too. > > > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list

"Martin Fuerderer" <MARTINFU@de.ibm.com> wrote in message news:mailman.278.1155721719.20706.informix-list@iiug.org... > Hi, > > the questions here are: > - how long does the snapshot take ? > - how often are checkpoints written ? > - what other write activity happens on disks ? > (e.g. foreground writes ... ?) > > If you start your snapshot (starting with the first disk), > and a new checkpoint happens writing on that first > disk after it was copied, but also on the other disks > before they are copied, then you have a physical > inconsistency. > > E.g. a blobspace BLOB is inserted. > The BLOB itself goes onto disk A that was already > copied, but the table (and the blob structure) are on > disk B that still is to be backed up. Then you will > later have a missing BLOB, because that BLOB > in the blobspace on disk A didn't make it onto the > snapshot. Whereas IDS cannot detect at the time > that this snapshot is inconsistent - it would not even > know that a snapshot was done. > This is just one example, you can construct others ... > > I'm not sure how "clever" a SAN and its snapshot > functionality are, but I certainly would not trust it to > detect and correctly handle examples like that above. > > And certainly I would not rely on the assumption > that "the snapshot is so fast that it is done before > any writes can introduce inconsistencies". > > If you insist in trying your method (without blocking), > then I advice you to run extensive "oncheck" commands > after the restore. If it finds inconsistencies, then you > simply do the whole snapshot exercise again ... > that is if the system is still around for that. Implicit - and perhaps I should have been explicit - in the whole thrust of my argument is that the SAN must have "consistency group" functionality: ie that within the group of LUNs defined as belonging to the database snap group, all the LUNs are "snapped" to the same point in time.