Problem with ER replication
Posted in 2013
Topics: High Availability & Replication, Platform-Specific Issues
All, Informix V11.70.FC2 Sun Solaris 5.10 I have an ER environment with 11 instances involved. 10 instances feed 1 instance. The 10 instances are on one machine and the 11th on another. Last night the main source instance ( and the notional master instance ) was taken offline. When the instance was restarted for some reason replication only recognised that instance on that machine. All the other instances involved recognise all other instances and the one started but it is marked as down. To clarify the server name for the stopped instance called euromarc_rep_group ( historical reasons ). When I run cdr list server on this instance I get: uxpifx04:/home/informix $ cdr list server SERVER ID STATE STATUS QUEUE CONNECTION CHANGED ----------------------------------------------------------------------- euromarc_rep_group 10 Active Local 0 When I run it on any of the other participating instances I get: uxpifx04:/home/informix $ cdr list server SERVER ID STATE STATUS QUEUE CONNECTION CHANGED ----------------------------------------------------------------------- admin_rep_group 110 Active Connected 0 Aug 10 09:53:31 agd_rep_group 40 Active Local 0 bo_rep_group 30 Active Connected 0 Aug 10 09:52:41 boa_rep_group 120 Active Connected 0 Aug 10 09:52:53 cms_rep_group 50 Active Connected 0 Aug 10 09:52:41 euromarc_rep_group 10 Active Failed 0 Sep 4 09:19:45 mp_rep_group 20 Active Connected 0 Aug 10 09:52:41 nice_rep_group 170 Active Connected 0 Aug 10 09:52:41 pub_rep_group 140 Active Connected 0 Aug 10 09:52:41 rcd_rep_group 80 Active Connected 0 Aug 10 09:52:41 rcdrec_rep_group 130 Active Connected 0 Aug 10 09:53:18 search_rep_group 90 Active Connected 0 Aug 10 09:52:41 So my question is how to get this environment working again? Is it better to remove replication completely from all instances and start again or can I try and get the euromarc_rep_group back into ER? If so what commands can I use. Regards Andy Grantham
Hello. It seems you have some network issue between your machines, or even an sqlhosts modification, that might be letting ER unable to see the two machines. You could start trying to do a simple "cdr connect", locally, to the remote instance, and see what is happening. If the message is not clearly enough, please post it here. Regards. Alexandre Marini IBM Informix Certified Professional v10 / v11.50 / v11.70 / v12.10 IBM Information Management Informix Technical Professional IBM Infosphere DataStage Technical Professional Informix Senior DBA - Orizon Brasil BRIUG website administrator Informix independent consultant > To: ids@iiug.org > From: agrantha@hotmail.com > Subject: Problem with ER replication [31349] > Date: Wed, 4 Sep 2013 04:23:07 -0400 > > All, > > Informix V11.70.FC2 > Sun Solaris 5.10 > > I have an ER environment with 11 instances involved. 10 instances feed 1 > instance. The 10 instances are on one machine and the 11th on another. Last > night the main source instance ( and the notional master instance ) was taken > offline. When the instance was restarted for some reason replication only > recognised that instance on that machine. All the other instances involved > recognise all other instances and the one started but it is marked as down. To > clarify the server name for the stopped instance called euromarc_rep_group ( > historical reasons ). When I run cdr list server on this instance I get: > > uxpifx04:/home/informix $ cdr list server > SERVER ID STATE STATUS QUEUE CONNECTION CHANGED > ----------------------------------------------------------------------- > euromarc_rep_group 10 Active Local 0 > > When I run it on any of the other participating instances I get: > > uxpifx04:/home/informix $ cdr list server > SERVER ID STATE STATUS QUEUE CONNECTION CHANGED > ----------------------------------------------------------------------- > admin_rep_group 110 Active Connected 0 Aug 10 09:53:31 > agd_rep_group 40 Active Local 0 > bo_rep_group 30 Active Connected 0 Aug 10 09:52:41 > boa_rep_group 120 Active Connected 0 Aug 10 09:52:53 > cms_rep_group 50 Active Connected 0 Aug 10 09:52:41 > euromarc_rep_group 10 Active Failed 0 Sep 4 09:19:45 > mp_rep_group 20 Active Connected 0 Aug 10 09:52:41 > nice_rep_group 170 Active Connected 0 Aug 10 09:52:41 > pub_rep_group 140 Active Connected 0 Aug 10 09:52:41 > rcd_rep_group 80 Active Connected 0 Aug 10 09:52:41 > rcdrec_rep_group 130 Active Connected 0 Aug 10 09:53:18 > search_rep_group 90 Active Connected 0 Aug 10 09:52:41 > > So my question is how to get this environment working again? Is it better to > remove replication completely from all instances and start again or can I try > and get the euromarc_rep_group back into ER? If so what commands can I use. > > Regards > > Andy Grantham > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. >
The syntax is: cdr connect server [remote_servername] Hope it helps. Regards. Alexandre Marini IBM Informix Certified Professional v10 / v11.50 / v11.70 / v12.10 IBM Information Management Informix Technical Professional IBM Infosphere DataStage Technical Professional Informix Senior DBA - Orizon Brasil BRIUG website administrator Informix independent consultant > To: ids@iiug.org > From: alexandre@briug.org > Subject: RE: Problem with ER replication [31352] > Date: Wed, 4 Sep 2013 08:44:14 -0400 > > Hello. > It seems you have some network issue between your machines, or even an > sqlhosts modification, that might be letting ER unable to see the two > machines. > > You could start trying to do a simple "cdr connect", locally, to the remote > instance, and see what is happening. If the message is not clearly enough, > please post it here. > > Regards. > > Alexandre Marini > IBM Informix Certified Professional v10 / v11.50 / v11.70 / v12.10 > > IBM Information Management Informix Technical Professional > > IBM Infosphere DataStage Technical Professional > Informix Senior DBA - Orizon Brasil > BRIUG website administrator > Informix independent consultant > > > To: ids@iiug.org > > From: agrantha@hotmail.com > > Subject: Problem with ER replication [31349] > > Date: Wed, 4 Sep 2013 04:23:07 -0400 > > > > All, > > > > Informix V11.70.FC2 > > Sun Solaris 5.10 > > > > I have an ER environment with 11 instances involved. 10 instances feed 1 > > instance. The 10 instances are on one machine and the 11th on another. Last > > night the main source instance ( and the notional master instance ) was > taken > > offline. When the instance was restarted for some reason replication only > > recognised that instance on that machine. All the other instances involved > > recognise all other instances and the one started but it is marked as down. > To > > clarify the server name for the stopped instance called euromarc_rep_group ( > > historical reasons ). When I run cdr list server on this instance I get: > > > > uxpifx04:/home/informix $ cdr list server > > SERVER ID STATE STATUS QUEUE CONNECTION CHANGED > > ----------------------------------------------------------------------- > > euromarc_rep_group 10 Active Local 0 > > > > When I run it on any of the other participating instances I get: > > > > uxpifx04:/home/informix $ cdr list server > > SERVER ID STATE STATUS QUEUE CONNECTION CHANGED > > ----------------------------------------------------------------------- > > admin_rep_group 110 Active Connected 0 Aug 10 09:53:31 > > agd_rep_group 40 Active Local 0 > > bo_rep_group 30 Active Connected 0 Aug 10 09:52:41 > > boa_rep_group 120 Active Connected 0 Aug 10 09:52:53 > > cms_rep_group 50 Active Connected 0 Aug 10 09:52:41 > > euromarc_rep_group 10 Active Failed 0 Sep 4 09:19:45 > > mp_rep_group 20 Active Connected 0 Aug 10 09:52:41 > > nice_rep_group 170 Active Connected 0 Aug 10 09:52:41 > > pub_rep_group 140 Active Connected 0 Aug 10 09:52:41 > > rcd_rep_group 80 Active Connected 0 Aug 10 09:52:41 > > rcdrec_rep_group 130 Active Connected 0 Aug 10 09:53:18 > > search_rep_group 90 Active Connected 0 Aug 10 09:52:41 > > > > So my question is how to get this environment working again? Is it better to > > remove replication completely from all instances and start again or can I > try > > and get the euromarc_rep_group back into ER? If so what commands can I use. > > > > Regards > > > > Andy Grantham > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. >
Is master node restored from backup ? If so, is it possible that point in time restore was performed on the master node, or logical log restore was skipped ? Can you check number of rows in servdef table in syscdr database ? There should be a row for each ER server. Also can you check master server message log file to see if any error messages printed while restarting ER after server recovery ? Thanks & Regards, Nagaraju From: "Andrew Grantham" <agrantha@hotmail.com> To: ids@iiug.org, Date: 09/04/2013 03:23 AM Subject: Problem with ER replication [31349] Sent by: ids-bounces@iiug.org All, Informix V11.70.FC2 Sun Solaris 5.10 I have an ER environment with 11 instances involved. 10 instances feed 1 instance. The 10 instances are on one machine and the 11th on another. Last night the main source instance ( and the notional master instance ) was taken offline. When the instance was restarted for some reason replication only recognised that instance on that machine. All the other instances involved recognise all other instances and the one started but it is marked as down. To clarify the server name for the stopped instance called euromarc_rep_group ( historical reasons ). When I run cdr list server on this instance I get: uxpifx04:/home/informix $ cdr list server SERVER ID STATE STATUS QUEUE CONNECTION CHANGED ----------------------------------------------------------------------- euromarc_rep_group 10 Active Local 0 When I run it on any of the other participating instances I get: uxpifx04:/home/informix $ cdr list server SERVER ID STATE STATUS QUEUE CONNECTION CHANGED ----------------------------------------------------------------------- admin_rep_group 110 Active Connected 0 Aug 10 09:53:31 agd_rep_group 40 Active Local 0 bo_rep_group 30 Active Connected 0 Aug 10 09:52:41 boa_rep_group 120 Active Connected 0 Aug 10 09:52:53 cms_rep_group 50 Active Connected 0 Aug 10 09:52:41 euromarc_rep_group 10 Active Failed 0 Sep 4 09:19:45 mp_rep_group 20 Active Connected 0 Aug 10 09:52:41 nice_rep_group 170 Active Connected 0 Aug 10 09:52:41 pub_rep_group 140 Active Connected 0 Aug 10 09:52:41 rcd_rep_group 80 Active Connected 0 Aug 10 09:52:41 rcdrec_rep_group 130 Active Connected 0 Aug 10 09:53:18 search_rep_group 90 Active Connected 0 Aug 10 09:52:41 So my question is how to get this environment working again? Is it better to remove replication completely from all instances and start again or can I try and get the euromarc_rep_group back into ER? If so what commands can I use. Regards Andy Grantham ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.