Re: Replication Error : HDR,

In article <68tu49$852$1@gte1.gte.net>, Hitesh Dave <davehit@vsol.com> writes >Hi folks, > > We have informix ver 7.23 running on solaris 5.5.1. We have two >informix instances configured in data replication pair (HDR). > > When we tried to build new indexes on couple of tables (containing 2 >million rows) on the primary instances, we got the following errors in >online.log. The data replication stopped on primary. The online log >messages indicate that there was an error while replicating the new >index. And it said to recreate the index on the primary instance. > > What we are doing right now is reload the secondary from the level 0 >archive of the primary. > > Can anybody throw more light on the following error messages. > >thanks for any help >hitesh dave > >Tue Jan 6 06:21:29 1998 > >06:21:29 Event alarms enabled. ALARMPROG = >'/usr/informix/etc/log_full.sh' >06:21:35 DR: DRAUTO is 0 (Off) >06:21:38 INFORMIX-OnLine Initialized -- Shared Memory Initialized. >06:21:38 DR: Trying to connect to secondary server ... >06:21:41 DR: Primary server connected >06:21:41 Physical Recovery Started. >06:21:41 Physical Recovery Complete: 0 Pages Restored. >06:21:41 Logical Recovery Started. >06:21:45 Logical Recovery Complete.> 0 Committed, 0 Rolled Back, 0 Open, 0 Bad Locks > >06:21:45 oninit: Invalid or missing name for Subsystem Staging >BLOBspace.> Check the ONCONFIG paramater STAGEBLOB either it is not in your onconfig or the value is wrong. >06:21:45 Dataskip is now OFF for all dbspaces >06:21:46 On-Line Mode >06:21:46 Checkpoint Completed: duration was 0 seconds. >06:22:13 DR: Sending log 1761, size 10000 pages, 66.67 percent used >06:22:13 Logical Log 1761 Complete. >06:22:20 Process exited with return code 139: /bin/sh /bin/sh -c >/usr/informix/e>tc/log_full.sh 2 23 "Logical Log 1761 Complete." "Logical Log 1761 >Complete." ??? ALARMPROGRAM is set to $INFORMIXDIR/etc/log_full.sh and something in that shell script exiting with error 139. >06:22:30 DR: Sending log 1762 (current), size 10000 pages, 0.00 percent>used >06:22:30 DR: Sending Logical Logs Completed >06:23:02 DR: Primary server operational >06:23:02 DR: LBU_PRESERVE turned on while primary >06:23:02 Checkpoint Completed: duration was 0 seconds. >06:23:37 DR: Receive error >06:23:37 Assert Failed: WARNING! Cannot send index >'htrn:"informix".trn_rx_info->Key#2' to Secondary >06:23:37 Who: Session(23, informix@batman, 0, 287435356) > Thread(37, dr_btsend, 111fc658, 3) Here the dr_btsend thread (Data Replication B-Tree Send thread) encountered an error. Possibly an Informix Bug. >06:23:37 Results: Index will be unusable on Secondary >06:23:37 Action: Drop and recreate on Primary >06:23:37 See Also: /tmp/af.252248 What does the file /tmp/af.252248 say? >06:23:37 DR: Send Btree error It has trouble sending the B-Tree i.e. the index to the secondary. >06:23:38 DR: Turned off on primary server >06:23:38 Checkpoint Completed: duration was 0 seconds. >06:23:38 DR: Cannot connect to secondary server >06:23:50 DR: Primary server connected >06:23:51 DR: Sending log 1762 (current), size 10000 pages, 0.03 percent>used >06:23:51 DR: Sending Logical Logs Completed >06:24:14 DR: Primary server operational >06:24:14 DR: LBU_PRESERVE turned on while primary >06:24:14 Checkpoint Completed: duration was 0 seconds. >06:30:24 DR: ping timeout This means the secondary did not respond. The Primary and Secondaty Servers periodically 'ping' one another to make sure they are both alive. This is how the secondary can automatically take over if the primary fails. >06:30:24 DR: Receive error >06:30:24 Assert Failed: WARNING! Cannot send index >'htrn:"informix".transaction->Key#3' to Secondary >06:30:24 Who: Session(33, informix@batman, 0, 287435648) > Thread(64, dr_btsend, 111fca98, 5) >06:30:24 Results: Index will be unusable on Secondary >06:30:24 Action: Drop and recreate on Primary >06:30:24 See Also: /tmp/af.4023e0 What does the file /tmp/af.4023e0 say? >06:30:24 DR: Send Btree error >06:30:26 DR: Turned off on primary server Database replication was turned off on the primary. i.e. the primary got no response back from the secondary. >06:30:27 Checkpoint Completed: duration was 0 seconds. >06:30:27 DR: Cannot connect to secondary server YeP! The secondary is not replying. What does the online.log for the secondary say. Try running netstat -i. How many Ierrs (Input errors) and Oerrs (output errors) are you getting, >15 indicates a possible network problem. Also what percentage of collisions are you getting on that network? >06:30:38 DR: Primary server connected >06:30:38 DR: Receive error >06:30:38 DR: Failure recovery error (2) >06:30:39 DR: Turned off on primary server >06:30:40 Checkpoint Completed: duration was 0 seconds. >06:30:40 DR: Cannot connect to secondary server >06:30:51 DR: Primary server connected >06:30:51 DR: Receive error >06:30:51 DR: Failure recovery error (2) >06:30:52 DR: Turned off on primary server The primary keeps connecting and then not receiveing a reply from the secondary. Also check the UNIX error log (/usr/adm/messages or /var/adm/messages or //var/deamon/log/messages). Look in /etc/syslog.conf for the correct filename). Semething should have been syslog'ed by UNIX. I suspect network errors or the secondary died. What does onstat -g ath on the secondary say at the point of failure? >06:30:53 Checkpoint Completed: duration was 0 seconds. >06:30:53 DR: Cannot connect to secondary server >06:31:04 DR: Primary server connected >06:31:04 DR: Receive error >06:31:04 DR: Failure recovery error (2) >06:31:05 DR: Turned off on primary server >06:31:06 Checkpoint Completed: duration was 0 seconds. >06:31:06 DR: Cannot connect to secondary server >06:31:17 DR: Primary server connected >06:31:17 DR: Receive error >06:31:17 DR: Failure recovery error (2) >06:31:18 DR: Turned off on primary server -- David Williams Maintainer of the Informix FAQ Primary site (Beta Version) http://www.smooth1.demon.co.uk Official site http://www.iiug.org/techinfo/faq/faq_top.html I see you standin', Standin' on your own, It's such a lonely place for you, For you to be If you need a shoulder, Or if you need a friend, I'll be here standing, Until the bitter end... So don't chastise me Or think I, I mean you harm... All I ever wanted Was for you To know that I care