Re: How to failback in HDR?
Posted in 2011
On Jan 20, 3:39 pm, <moss...@wellsfargo.com> wrote:
> > My DR_AUTO is set to 0, manual. I'm able to get replication going
> > from my primary server to my secondary. I'm able to effectively
> > failover to my secondary, changing secondary informixserver mode to
> > standalone, and successfully pointing users' applications (with
> > attendant appserver/webserver config changes) to the secondary. But I
> > haven't solved the conundrum of failback: I run simultaneous ontape
> > archive-restore across the network from original secondary to
> > primary. The restore completes without error, leaving the original
> > primary in recovery mode, with original secondary in standone, multi-
> > user mode. How do I proceed from here (manuals don't cover this exact
> > case)? I've attempted a number of different sequences of onmode
> > commands on both hosts, but to no avail -- replication doesn't
> > recommence with original primary as primary and original secondary as
> > secondary, and occasionally even crashes the secondary, leaving behind
> > nasty flotsam of shared memory segments.
>
> > I could simply bring the primary to multi-user mode after the failback
> > restore, then execute yet another ontape archive-restore from primary
> > back to secondary and recommence replication. But this strikes me as
> > unnecessary since after failback restore to the original primary the
> > two informixservers are nearly synchronized. Gotta be a better way.
>
> > What are the precise steps, in sequence, for each informixserver after
> > failback restore with primary in recovery and secondary in multi-user
> > mode?
>
> > Using IDS 11.50.FC4 on HPUX 11.23.
>
> Usually no need for new archives at all. Here's how we handle switchovers:
>
> 1. orig_primary to quiescent mode; roll to new log; force ckpt; shutdown
> 2. orig_secondary to standard/online mode << you can point the apps to this server now
> 3. bring orig_primary back up via "oninit -PHY" << does NOT go thru logical recovery
> 4. switch orig_secondary to primary mode
> 5. switch orig_primary to secondary mode (the "Clearing Logs" part may take a few minutes)
> 6. when replication is operational, you now have replication active in the *reverse* direction
>
> Perform necessary fixes / maintenance
>
> When ready to switch back:
> 7. orig_secondary to quiescent mode; roll to new log; force ckpt; shutdown
> 8. orig_primary to standard/online mode << you can point the apps to this server now
> 9. bring orig_secondary back up via "oninit -PHY" << does NOT go thru logical recovery
> 10. switch orig_primary to primary mode
> 11. switch orig_secondary to secondary mode (the "Clearing Logs" part may take a few minutes)
> 12. when replication is operational, you now have replication active in the *normal* direction
>
> A couple of keys to making this all work:
> -- There cannot be any checkpoints on the server that has been primary, after it loses connection to the secondary. That is why we just do the shutdown without switching the primary to standard mode.
> -- The special "oninit -PHY" brings up the instance to "recovery mode", i.e., physical recovery only without doing any logical recovery.
>
> HTH,
> Paul Mosser
Thank you all for replying. Referring to the vendor references wasn't
helpful, as I said in original post, since this case (failback after
failover) isn't described in TFMs. But I now have a working SOP by
following Mosser's steps in detail. Thanks Paul. Remarkable that
product lead software engineer (Pruitt) for IDS replication still
participates in this forum. Here are the regurgitated steps (YMMV):
This procedure assumes that HDR had been in effect from a
primary informixserver to a secondary; that a failover was necessary
to
the secondary informixserver, establishing it as a standard server;
and that the original primary informixserver is available and ready
to re-establish the original replication scheme.
1) Archive from the original secondary informixserver and restore to
the
original primary informixserver. Use the archive_restore.ksh script,
which simultaneously executes an ontape archive from a source to a
target informixserver using the called script ontape_HDR_restore.ksh
with ssh. The process takes about four hours depending on other
processing on hosts, network usage, etc.
2) Check the Informix messages logs and the script logs. When the
archive-restore is complete, the secondary will still be in standard
mode; the primary will be in recovery mode.
4) Start replication in reverse from original secondary to original
primary. On the original
secondary, execute:
onmode -d primary <original_primary_informixserver>
On the original primary, exeute:
onmode -d secondary <original_secondary_informixserver>
Wait until replication is underway. This will be indicated by the
status message when "onstat -" is executed at the command line for
each informixserver.
4) Invert the replication flow back again. On original secondary,
take the
instance to quiescent mode:
onmode -s
Roll to a new log:
onmode -l
Force a checkpoint:
onmode -c
Shutdown the informixserver:
cd ${CHUNK_BASE_PATH}
onmode -ky
5) Take the original primary informixserver to multi-user mode:
onmode -m
6) On the original secondary informixserver, while still in $
{CHUNK_BASE_PATH},
bring the instance back up, but avoid logical recovery:
oninit -PHY
7) On the original primary informixserver, turn on replication:
onmode -d primary <secondary_informixserver>
8) On the original secondary informixserver, turn on replication:
onmode -d secondary <primary_informixserver>
9) Observe the server statuses by executing "onstat -" or onstat -g
dri. The primary
should indicate that it is online as primary. The secondary should
indicate that it is secondary in read-only mode.