Re: HDR questions
A user wanted to offload work from an HDR primary: he hoped to onunload a database from the HDR secondary (temporarily suspending replication) to refresh a copy of production, avoiding shipping a 20GB level-0 archive over a slow T1 link. Madison Pruet said that approach isn't likely to be implemented for technical reasons, but suggested piping an STDIO backup through compression (ontape -s -L 0 -t STDIO | gzip) to cut the data transferred. The poster accepted this; another user posted a bash script that pipes ontape output through gzip/rsh to gunzip and ontape -p on the remote machine to build the replicate, and a third confirmed using the technique in production.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
"bozon" <curtis@crowson1.com> wrote in message
news:1153759637.436580.93640@s13g2000cwa.googlegroups.com...
> I always seem to crash into reality when I have flights of fancy.
>
> If you didn't have any of the afore mentioned features could you allow
> it? I know this limits the capability but many sites don't use those
> features.
>
I would also echo this. We do not use any of the mentioned features, and
would like to be able to capture a consistent version of a particular
database on a secondary of an HDR pair.
Why? you ask?
The production server needs to have maximum availability. For legacy
reasons that I had nothing to do with, in addition to the production
database on the primary, there is also a copy of the production database
that is used by a small group of researchers. It does not need to be
synchronized with the production database in real time, but occasionally
needs to be refreshed with a current copy of production. To minimize any
contention for resources on the primary, I'd like to be able to 'onunload'
the secondary's copy of the production database, and then 'onload' it to the
primary as the research database.
In the past, before we started using HDR, I'd use ontape to capture a backup
of the entire instance (which is much larger than the production database
due to the copy plus several other legacy databases on the same instance),
then 'ontape -r' it to an identical sister machine, then 'onunload' the
production database, then 'onload' it back on the production server as the
research database. The advantage of this circuitous route is that I
refreshed the reseach database from the production database with 0 downtime
on production.
I'd like to accomplish the same thing (0 production downtime) in the HDR
environment, without having to re-establish HDR from the beginning. By this
I mean, I want to avoid having to truck a 20 GB level 0 archive across a
limited bandwidth WAN (T1) that connects the HDR primary and secondary. (It
is much less time consuming to transmit only the result of the 'onunload' [5
GB] back to the primary.) It would help greatly if I could temporarily
suspend replication while onunloading the secondary, then pick it up again
when onunload finishes.
Regards,
DG
David E. Grove wrote:
> "bozon" <curtis@crowson1.com> wrote in message
> news:1153759637.436580.93640@s13g2000cwa.googlegroups.com...
>> I always seem to crash into reality when I have flights of fancy.
>>
>> If you didn't have any of the afore mentioned features could you allow
>> it? I know this limits the capability but many sites don't use those
>> features.
>>
>
> I would also echo this. We do not use any of the mentioned features, and
> would like to be able to capture a consistent version of a particular
> database on a secondary of an HDR pair.
David,
I understand the reasons that this would be desirable, but there are
some technical reasons that I doubt that this specific solution will be
implemented as you describe. That said, I also think that we have some
solutions in the works which will address your basic requirements.
M.P.
>
> Why? you ask?
>
> The production server needs to have maximum availability. For legacy
> reasons that I had nothing to do with, in addition to the production
> database on the primary, there is also a copy of the production database
> that is used by a small group of researchers. It does not need to be
> synchronized with the production database in real time, but occasionally
> needs to be refreshed with a current copy of production. To minimize any
> contention for resources on the primary, I'd like to be able to 'onunload'
> the secondary's copy of the production database, and then 'onload' it to the
> primary as the research database.
>
> In the past, before we started using HDR, I'd use ontape to capture a backup
> of the entire instance (which is much larger than the production database
> due to the copy plus several other legacy databases on the same instance),
> then 'ontape -r' it to an identical sister machine, then 'onunload' the
> production database, then 'onload' it back on the production server as the
> research database. The advantage of this circuitous route is that I
> refreshed the reseach database from the production database with 0 downtime
> on production.
>
> I'd like to accomplish the same thing (0 production downtime) in the HDR
> environment, without having to re-establish HDR from the beginning. By this
> I mean, I want to avoid having to truck a 20 GB level 0 archive across a
> limited bandwidth WAN (T1) that connects the HDR primary and secondary. (It
> is much less time consuming to transmit only the result of the 'onunload' [5
> GB] back to the primary.) It would help greatly if I could temporarily
> suspend replication while onunloading the secondary, then pick it up again
> when onunload finishes.
>
> Regards,
>
> DG
>
>
>
David E. Grove wrote:
> "bozon" <curtis@crowson1.com> wrote in message
> news:1153759637.436580.93640@s13g2000cwa.googlegroups.com...
>> I always seem to crash into reality when I have flights of fancy.
>>
>> If you didn't have any of the afore mentioned features could you allow
>> it? I know this limits the capability but many sites don't use those
>> features.
>>
>
> I would also echo this. We do not use any of the mentioned features, and
> would like to be able to capture a consistent version of a particular
> database on a secondary of an HDR pair.
>
> Why? you ask?
David,
A technique that you could use to avoid having to transport so much
stuff would be to take an STDIO backup and then use the following....
ontape -s -L 0 -F | gzip > backup_tape....
I suspect that the compressed backup would be much smaller than the
standard backup. That would minimize the amount of stuff which had to
be transfer accross the network.
M.P.
>
> The production server needs to have maximum availability. For legacy
> reasons that I had nothing to do with, in addition to the production
> database on the primary, there is also a copy of the production database
> that is used by a small group of researchers. It does not need to be
> synchronized with the production database in real time, but occasionally
> needs to be refreshed with a current copy of production. To minimize any
> contention for resources on the primary, I'd like to be able to 'onunload'
> the secondary's copy of the production database, and then 'onload' it to the
> primary as the research database.
>
> In the past, before we started using HDR, I'd use ontape to capture a backup
> of the entire instance (which is much larger than the production database
> due to the copy plus several other legacy databases on the same instance),
> then 'ontape -r' it to an identical sister machine, then 'onunload' the
> production database, then 'onload' it back on the production server as the
> research database. The advantage of this circuitous route is that I
> refreshed the reseach database from the production database with 0 downtime
> on production.
>
> I'd like to accomplish the same thing (0 production downtime) in the HDR
> environment, without having to re-establish HDR from the beginning. By this
> I mean, I want to avoid having to truck a 20 GB level 0 archive across a
> limited bandwidth WAN (T1) that connects the HDR primary and secondary. (It
> is much less time consuming to transmit only the result of the 'onunload' [5
> GB] back to the primary.) It would help greatly if I could temporarily
> suspend replication while onunloading the secondary, then pick it up again
> when onunload finishes.
>
> Regards,
>
> DG
>
>
>
"Madison Pruet" <mpruet@comcast.net> wrote in message
news:44C91399.8060400@comcast.net...
>
> David,
>
> A technique that you could use to avoid having to transport so much
> stuff would be to take an STDIO backup and then use the following....
>
>
> ontape -s -L 0 -F | gzip > backup_tape....>
Thank you for the excellent suggestion. Sometimes I can't even see my own
face in the mirror!
DG
Below is shell script I use to setup a replicate in 10. The only thing
that I would change in your case is I would always gzip and gunzip the
file as it went across the network because of your slow wan. I only do
that in my script when I "tee" the output to a file, which I do when I
want to take a backup while I am setting up the replicate. Boy, backing
up to standard out is fun.
This was a very quick script and is bound to not be rugged. It works
well for me on tru64. The machines need to trust each other so that you
can rsh between them of course.
just add a step like this in the if clause after the if [ $# -lt 2] ;
then clause:
elif [ $# -eq 2 ] ; then
createFileSrc="gzip -c | "
createFileTgt="gunzip -c"
I think the above addition is correct, but I haven't tested it.
#!/usr/local/bin/bash
#set -x # for debugging.
# Curtis Crowson 2006-07-26
# This is a copy of the extRestore script that will do replicated
restores. These probably should be the same script with a -p
# option or some such thing but I am going to do a quick and dirty
script instead.
# CLC quick script to do replicated restore. Doesn't have many features
or much error checking. I suspect that it will grow.
function usage {
cat <<EOF 1>&2
$( if [ $# -eq 1 ] ; then printf "Error: %s\\n" "$1" ; fi )
Usage: $(basename $0) <target-machine> <target-database>
<optional-file>
This script will restore and create a replicate of the current
instance to the target instance on the specified machine.
<optional-file> will be a gzipped file that the backup will be saved
in while the replication is being created.
EOF
exit 1
}
createFileSrc=""
createFileTgt=""
if [ $# -lt 2 ] ; then
usage "not enough parameters."
elif [ $# -eq 3 ] ; then
#Make sure file ends in .gz only once. remove it if it exists and
then add it on.
fileName=${3%%\\.gz*}.gz
createFileSrc="gzip -c | tee $fileName "
createFileTgt="gunzip -c"
elif [ $# -gt 3 ] ; then
usage "too many parameters."
fi
ontape -s -L 0 -t STDIO | \\
eval $createFileSrc | \\
rsh $1 "( \\ export INFORMIXDIR=/usr/local/informix ; \\
export PATH=$PATH:$INFORMIXDIR/bin ; \\
export INFORMIXSERVER=$2 ; \\
export ONCONFIG=onconfig.$2 ; \\ $createFileTgt | \\
ontape -p -t STDIO \\
)"
Madison Pruet wrote:
> David E. Grove wrote:
> > "bozon" <curtis@crowson1.com> wrote in message
> > news:1153759637.436580.93640@s13g2000cwa.googlegroups.com...
> >> I always seem to crash into reality when I have flights of fancy.
> >>
> >> If you didn't have any of the afore mentioned features could you allow
> >> it? I know this limits the capability but many sites don't use those
> >> features.
> >>
> >
> > I would also echo this. We do not use any of the mentioned features, and
> > would like to be able to capture a consistent version of a particular
> > database on a secondary of an HDR pair.
> >
> > Why? you ask?
>
> David,
>
> A technique that you could use to avoid having to transport so much
> stuff would be to take an STDIO backup and then use the following....
>
>
> ontape -s -L 0 -F | gzip > backup_tape....>
> I suspect that the compressed backup would be much smaller than the
> standard backup. That would minimize the amount of stuff which had to
> be transfer accross the network.
>
> M.P.
> >
> > The production server needs to have maximum availability. For legacy
> > reasons that I had nothing to do with, in addition to the production
> > database on the primary, there is also a copy of the production database
> > that is used by a small group of researchers. It does not need to be
> > synchronized with the production database in real time, but occasionally
> > needs to be refreshed with a current copy of production. To minimize any
> > contention for resources on the primary, I'd like to be able to 'onunload'
> > the secondary's copy of the production database, and then 'onload' it to the
> > primary as the research database.
> >
> > In the past, before we started using HDR, I'd use ontape to capture a backup
> > of the entire instance (which is much larger than the production database
> > due to the copy plus several other legacy databases on the same instance),
> > then 'ontape -r' it to an identical sister machine, then 'onunload' the
> > production database, then 'onload' it back on the production server as the
> > research database. The advantage of this circuitous route is that I
> > refreshed the reseach database from the production database with 0 downtime
> > on production.
> >
> > I'd like to accomplish the same thing (0 production downtime) in the HDR
> > environment, without having to re-establish HDR from the beginning. By this
> > I mean, I want to avoid having to truck a 20 GB level 0 archive across a
> > limited bandwidth WAN (T1) that connects the HDR primary and secondary. (It
> > is much less time consuming to transmit only the result of the 'onunload' [5
> > GB] back to the primary.) It would help greatly if I could temporarily
> > suspend replication while onunloading the secondary, then pick it up again
> > when onunload finishes.
> >
> > Regards,
> >
> > DG
> >
> >
> >
"David E. Grove" <david_grove@correct.state.ak.us> wrote in message
news:12ci5jleu580301@corp.supernews.com...
>
> "Madison Pruet" <mpruet@comcast.net> wrote in message
> news:44C91399.8060400@comcast.net...
>>
>> David,
>>
>> A technique that you could use to avoid having to transport so much
>> stuff would be to take an STDIO backup and then use the following....
>>
>>
>> ontape -s -L 0 -F | gzip > backup_tape....>>
>
> Thank you for the excellent suggestion. Sometimes I can't even see my own
> face in the mirror!
We use this technique to good effect, using a live database in Leeds to
referesh nightly several support and test ones in London, some 450km to the
south. Let me know if you'd like the scripts.
rgds
Neil