RE: Replication failure Q
Posted in 2009
Topics: High Availability & Replication, Backup & Restore, Installation, Setup & Upgrades, Server Administration, Platform-Specific Issues
Well the water rises. Now while both servers are in standard mode I want to
benchmark the new disk drive with a point in time restore to get some data
that caused us a problem before and I come to a new problem! Informix will
not let me restore!
It gets past the roodbs than tries to start Informix BUT it tries to connect
to itself as the secondary and all the restores are not done yet!!!!
We were working fine until fc2W5 was installed. We used to work fine on
plain 11.10fc2.
So now I can't get my secondary back online! Since fc2W5 patches have
smashed my HDR can anyone say if 11.50 works HDR any better? Which patch
level is the best with HDR? I can't back off W5 since one of my tables
causes plan fc2 to assert itself.
I have used HDR very successfully since 1994. It was very straight forward
and easy to understand. I don't see why an onbar -r -p would even try to
connect to a secondary server much less itself? (It was the secondary before
I went standard).
1) onmode -d standard on the secondary no longer works! What happens if this
were a real disaster? I need my secondary to come online as standard!
2) after doing an onmode -d standard on the primary I was able to do a
onmode -d standard on the secondary.BOTH servers were in standard mode.
I put in the new disk drive...
3) onbar -r -p on the secondary fails due to the initialization wanting to
connect to a secondary server (itself).
4) DRAUTO=0 due some things I did not like about Auto failover in an earlier
release.
Looks like DRAUTO=0 is tried to do some sort of Automatic recovery or
something.
So I am currently stuck not being able to restore to the secondary. I hope
the primary stays up!
,,,Tim
Tim Ertl
413-442-9000 x6211
This email was sent with Microsoft Outlook. If you see a winmail.dat or have
problems reading this email please let me know and I will resend using a RFC
compliant mail program.
Thanks,,,Tim
-----Original Message-----
From: Tim Ertl [mailto:tim@lmrgroup.com]
Sent: Saturday, March 28, 2009 10:57
To: 'ids@iiug.org'
Subject: Replication failure Q
I just discovered that IDS 11.10 replication has a significantly different
operational feature than the old 9 days. In particular when the PRIMARY
fails we used to be able to issue an "onmode -d standard" to put the
secondary into production. We have done this a number of times on 9.3.
When we installed 11.10 we can not longer do this. We get a messages after
doing the Onmode -d standard (on secondary) saying:
DR: Unable to change server type
In the online log I see:
DR: DR should be turned off for secondary to standard change.
This message is amusing since I think onmode -d standard is how you turn off
replication. At least in the past.
Today I researched and played. I see the onmode -d operations for the
secondary's. Then I discovered that I can do the onmode -d standard on the
Primary and THEN do the onmode -d standard on the secondary. But that is
pretty much useless in the event of a real hardware failure on the primary.
I do not see any new "turn off replication" commands but I see a mess of
other new commands for RSS and other fun stuff. How do I promote the
secondary into standard mode if the primary is down? The 11.10 manual seems
to imply about the same thing as the 9.3 manual so I am confused.
If no-one has any ideas I will report this next week to Informix Support as
a possible bug.
IDS 11.10fc1W5+ on Sun Solaris 10.
Thanks,,,
,,,Tim
Tim Ertl
413-442-9000 x6211
This email was sent with Microsoft Outlook. If you see a winmail.dat or have
problems reading this email please let me know and I will resend using a RFC
compliant mail program.
Thanks,,,Tim
What is in the secondary message log file after the physical restore?
=
"Tim Ertl" =
<tim@lmrgroup.com =
> =
To
Sent by: ids@iiug.org =
ids-bounces@iiug. =
cc
org =
Subj=
ect
RE: Replication failure Q [1535=
9]
03/28/2009 01:35 =
PM =
=
=
Please respond to =
ids@iiug.org =
=
=
Well the water rises. Now while both servers are in standard mode I wan=
t to
benchmark the new disk drive with a point in time restore to get some d=
ata
that caused us a problem before and I come to a new problem! Informix w=
ill
not let me restore!
It gets past the roodbs than tries to start Informix BUT it tries to
connect
to itself as the secondary and all the restores are not done yet!!!!
We were working fine until fc2W5 was installed. We used to work fine on=
plain 11.10fc2.
So now I can't get my secondary back online! Since fc2W5 patches have
smashed my HDR can anyone say if 11.50 works HDR any better? Which patc=
h
level is the best with HDR? I can't back off W5 since one of my tables
causes plan fc2 to assert itself.
I have used HDR very successfully since 1994. It was very straight forw=
ard
and easy to understand. I don't see why an onbar -r -p would even try t=
o
connect to a secondary server much less itself? (It was the secondary
before
I went standard).
1) onmode -d standard on the secondary no longer works! What happens if=
this
were a real disaster? I need my secondary to come online as standard!
2) after doing an onmode -d standard on the primary I was able to do a
onmode -d standard on the secondary.BOTH servers were in standard mode.
I put in the new disk drive...
3) onbar -r -p on the secondary fails due to the initialization wanting=
to
connect to a secondary server (itself).
4) DRAUTO=3D0 due some things I did not like about Auto failover in an
earlier
release.
Looks like DRAUTO=3D0 is tried to do some sort of Automatic recovery or=
something.
So I am currently stuck not being able to restore to the secondary. I h=
ope
the primary stays up!
,,,Tim
Tim Ertl
413-442-9000 x6211
This email was sent with Microsoft Outlook. If you see a winmail.dat or=
have
problems reading this email please let me know and I will resend using =
a
RFC
compliant mail program.
Thanks,,,Tim
-----Original Message-----
From: Tim Ertl [mailto:tim@lmrgroup.com]
Sent: Saturday, March 28, 2009 10:57
To: 'ids@iiug.org'
Subject: Replication failure Q
I just discovered that IDS 11.10 replication has a significantly differ=
ent
operational feature than the old 9 days. In particular when the PRIMARY=
fails we used to be able to issue an "onmode -d standard" to put the
secondary into production. We have done this a number of times on 9.3.
When we installed 11.10 we can not longer do this. We get a messages af=
ter
doing the Onmode -d standard (on secondary) saying:
DR: Unable to change server type
In the online log I see:
DR: DR should be turned off for secondary to standard change.
This message is amusing since I think onmode -d standard is how you tur=
n
off
replication. At least in the past.
Today I researched and played. I see the onmode -d operations for the
secondary's. Then I discovered that I can do the onmode -d standard on =
the
Primary and THEN do the onmode -d standard on the secondary. But that i=
s
pretty much useless in the event of a real hardware failure on the prim=
ary.
I do not see any new "turn off replication" commands but I see a mess o=
f
other new commands for RSS and other fun stuff. How do I promote the
secondary into standard mode if the primary is down? The 11.10 manual s=
eems
to imply about the same thing as the 9.3 manual so I am confused.
If no-one has any ideas I will report this next week to Informix Suppor=
t as
a possible bug.
IDS 11.10fc1W5+ on Sun Solaris 10.
Thanks,,,
,,,Tim
Tim Ertl
413-442-9000 x6211
This email was sent with Microsoft Outlook. If you see a winmail.dat or=
have
problems reading this email please let me know and I will resend using =
a
RFC
compliant mail program.
Thanks,,,Tim
***********************************************************************=
********
Forum Note: Use "Reply" to post a response in the discussion forum.
=
Hi Madison! I hope to see you at IIUG again!
Only the rootdbs gets restored, then onbar attempts to bring up the instance
and fails. Oncheck -pr shows all of the stuff that was backed up from the
primary. So I think that is why the onbar startup attempts to connect to the
secondary. It looks like this data is not corrected by oninit before
starting the server? It actually thinks it is the primary?
rc1 is my primary normally and rc2 is my secondary.
This is the online.log output from my secondary during the restore.
,,,Tim
12:17:30 IBM Informix Dynamic Server Version 11.10.FC2W5 Software Serial
Number AAA#B000000
12:17:31 IBM Informix Dynamic Server Initialized -- Shared Memory
Initialized.
12:17:31 DR: Reservation of the last logical log for log backup turned on
12:17:32 DR: Trying to connect to secondary server = rc2
12:17:34 DR: Turned off on primary server
12:17:34 DR: Local and Remote server type and/or last change(LC)
incompatible
[Local type: Primary, LC: Primary mode]
[Remote type: Primary, LC: ]
12:17:34 DR: Cannot start the Primary server when the paired server is of
incompatible (Primary or Standard) type. Please convert
the paired server to compatible (Secondary) type.
12:17:34 oninit: Fatal error in shared memory initialization
,,,Tim
Tim Ertl
413-442-9000 x6211
This email was sent with Microsoft Outlook. If you see a winmail.dat or have
problems reading this email please let me know and I will resend using a RFC
compliant mail program.
Thanks,,,Tim
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
Madison Pruet
Sent: Saturday, March 28, 2009 15:05
To: ids@iiug.org
Subject: RE: Replication failure Q [15360]
What is in the secondary message log file after the physical restore?
=
"Tim Ertl" =
<tim@lmrgroup.com =
> =
To
Sent by: ids@iiug.org =
ids-bounces@iiug. =
cc
org =
Subj=
ect
RE: Replication failure Q [1535=
9]
03/28/2009 01:35 =
PM =
=
=
Please respond to =
ids@iiug.org =
=
=
Well the water rises. Now while both servers are in standard mode I wan=
t to
benchmark the new disk drive with a point in time restore to get some d=
ata
that caused us a problem before and I come to a new problem! Informix w=
ill
not let me restore!
It gets past the roodbs than tries to start Informix BUT it tries to
connect
to itself as the secondary and all the restores are not done yet!!!!
We were working fine until fc2W5 was installed. We used to work fine on=
plain 11.10fc2.
So now I can't get my secondary back online! Since fc2W5 patches have
smashed my HDR can anyone say if 11.50 works HDR any better? Which patc=
h
level is the best with HDR? I can't back off W5 since one of my tables
causes plan fc2 to assert itself.
I have used HDR very successfully since 1994. It was very straight forw=
ard
and easy to understand. I don't see why an onbar -r -p would even try t=
o
connect to a secondary server much less itself? (It was the secondary
before
I went standard).
1) onmode -d standard on the secondary no longer works! What happens if=
this
were a real disaster? I need my secondary to come online as standard!
2) after doing an onmode -d standard on the primary I was able to do a
onmode -d standard on the secondary.BOTH servers were in standard mode.
I put in the new disk drive...
3) onbar -r -p on the secondary fails due to the initialization wanting=
to
connect to a secondary server (itself).
4) DRAUTO=3D0 due some things I did not like about Auto failover in an
earlier
release.
Looks like DRAUTO=3D0 is tried to do some sort of Automatic recovery or=
something.
So I am currently stuck not being able to restore to the secondary. I h=
ope
the primary stays up!
,,,Tim
Tim Ertl
413-442-9000 x6211
This email was sent with Microsoft Outlook. If you see a winmail.dat or=
have
problems reading this email please let me know and I will resend using =
a
RFC
compliant mail program.
Thanks,,,Tim
-----Original Message-----
From: Tim Ertl [mailto:tim@lmrgroup.com]
Sent: Saturday, March 28, 2009 10:57
To: 'ids@iiug.org'
Subject: Replication failure Q
I just discovered that IDS 11.10 replication has a significantly differ=
ent
operational feature than the old 9 days. In particular when the PRIMARY=
fails we used to be able to issue an "onmode -d standard" to put the
secondary into production. We have done this a number of times on 9.3.
When we installed 11.10 we can not longer do this. We get a messages af=
ter
doing the Onmode -d standard (on secondary) saying:
DR: Unable to change server type
In the online log I see:
DR: DR should be turned off for secondary to standard change.
This message is amusing since I think onmode -d standard is how you tur=
n
off
replication. At least in the past.
Today I researched and played. I see the onmode -d operations for the
secondary's. Then I discovered that I can do the onmode -d standard on =
the
Primary and THEN do the onmode -d standard on the secondary. But that i=
s
pretty much useless in the event of a real hardware failure on the prim=
ary.
I do not see any new "turn off replication" commands but I see a mess o=
f
other new commands for RSS and other fun stuff. How do I promote the
secondary into standard mode if the primary is down? The 11.10 manual s=
eems
to imply about the same thing as the 9.3 manual so I am confused.
If no-one has any ideas I will report this next week to Informix Suppor=
t as
a possible bug.
IDS 11.10fc1W5+ on Sun Solaris 10.
Thanks,,,
,,,Tim
Tim Ertl
413-442-9000 x6211
This email was sent with Microsoft Outlook. If you see a winmail.dat or=
have
problems reading this email please let me know and I will resend using =
a
RFC
compliant mail program.
Thanks,,,Tim
***********************************************************************=
********
Forum Note: Use "Reply" to post a response in the discussion forum.
=
****************************************************************************
***
Forum Note: Use "Reply" to post a response in the discussion forum.