ER error
Frank reported CDR/Enterprise Replication messages in the online log on a new 11.50.FC8 Linux (x86_64) setup whenever 'cdr start'/'cdr stop' was run: 'rqmSpoolHeader()/rqmSpoolTxn() failed ... queue control_send', 'Queuer: rqmQueueTxn failed on CNTRL', and 'GC: operation control message construction failed (error -1)' (once error 48, out of memory). Karl suggested the messages are normal around replication stop/start; Alexandre asked whether ER was rebuilt from scratch and then advised opening a PMR with IBM since no SQL error or AF file was produced. Frank confirmed it was a brand-new ER build and that replication itself works. No definitive cause or fix is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Experts, Any comments on the following ER errors in log file? You can ignore the warning of un-synced clock. Thanks, Frank Build Version: 11.50.FC8 Build Number: N198 Build Host: vidar Build OS: Linux 2.6.9-34.ELsmp Build Date: Fri Nov 12 21:58:33 CST 2010 GLS Version: glslib-4.50.FC11 [informix@maggie bin]$ uname -a Linux maggie 2.6.18-238.5.1.el5 #1 SMP Mon Feb 21 05:52:39 EST 2011 x86_64 x86_64 x86_64 GNU/Linux cdr stop: 19:31:02 Maximum server connections 2 19:31:02 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked 0, Plog used 61, Llog used 129 19:33:17 CDR connection to server lost, id 23, name <g_ngdcops> Reason: disconnect server 19:33:17 CDR connection to server lost, id 22, name <g_ncdcops> Reason: disconnect server 19:33:17 CDR NIF Shutdown: connections all shutdown. 19:33:17 CDR The NIF sub-component has shut down. 19:33:21 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x16/0x(nil)) status=-1 SQLERR=0 thread 2893 queue control_send 19:33:21 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x16/0x(nil)) status=-1 SQLERR=0 thread 2893 queue control_send 19:33:21 CDR Queuer: rqmQueueTxn (21/0/0x16/0x0) failed on CNTRL 19:33:21 CDR GC: operation control message construction failed (error -1). 19:33:26 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x17/0x(nil)) status=-1 SQLERR=0 thread 2894 queue control_send 19:33:26 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x17/0x(nil)) status=-1 SQLERR=0 thread 2894 queue control_send 19:33:26 CDR Queuer: rqmQueueTxn (21/0/0x17/0x0) failed on CNTRL 19:33:26 CDR GC: operation control message construction failed (error -1). 19:33:28 DDR Log Snooping - Shutdown 19:33:29 CDR shutdown complete cdr start: 19:35:01 CDR GC: catalog recovery begin 19:35:01 CDR GC: operation control message construction failed (error -1). 19:35:04 CDR GC: catalog recovery complete 19:35:04 CDR queuer initialization complete 19:35:04 CDR NIF listening on asf://g_nsofops 19:35:04 DDR Log Snooping - Snooping started in log 9 19:35:04 CDR NIF site: 21 <g_ngdcops> clocks differ by 11 seconds 19:35:04 Warning - The operating system time of day clock at the remote site differs from the local site. This can slow down the data-sync apply performance at the target server. This happens only if timestamp or stored procedure conflict resolution rules are being used. 19:35:04 CDR: Re-connected to server, id 23, name <g_ngdcops> 19:35:04 CDR NIF site: 21 <g_ncdcops> clocks differ by 11 seconds 19:35:04 Warning - The operating system time of day clock at the remote site differs from the local site. This can slow down the data-sync apply performance at the target server. This happens only if timestamp or stored procedure conflict resolution rules are being used. 19:35:04 CDR: Re-connected to server, id 22, name <g_ncdcops> --bcaec53961e2c1f5f304a25028c1
More interesting info. Any suggestions? Thanks, Frank [informix@maggie cdr]$ cdr finderr 48 48 out of memory 21:11:10 On-Line Mode 21:11:11 SCHAPI: Started dbScheduler thread. 21:11:11 Booting Language <spl> from module <> 21:11:11 Loading Module <SPLNULL> 21:11:12 SCHAPI: Started 2 dbWorker threads. 21:11:12 CDR Queuer: Could not open handle on queue control_send. 21:11:12 CDR GC: operation control message construction failed (error 48). 21:11:13 CDR GC: catalog recovery complete 21:11:13 CDR queuer initialization complete 21:11:13 CDR NIF listening on asf://g_nsofops 21:11:13 DDR Log Snooping - Snooping started in log 9 21:11:14 CDR NIF site: 21 <g_ngdcops> clocks differ by 12 seconds 21:11:14 Warning - The operating system time of day clock at the remote site differs from the local site. This can slow down the data-sync apply performance at the On Mon, May 2, 2011 at 3:38 PM, FRANK <yunyaoqu@gmail.com> wrote: > Experts, > > Any comments on the following ER errors in log file? You can ignore the > warning of un-synced clock. > > Thanks, > Frank > > Build Version: 11.50.FC8 > Build Number: N198 > Build Host: vidar > Build OS: Linux 2.6.9-34.ELsmp > Build Date: Fri Nov 12 21:58:33 CST 2010 > GLS Version: glslib-4.50.FC11 > > [informix@maggie bin]$ uname -a > Linux maggie 2.6.18-238.5.1.el5 #1 SMP Mon Feb 21 05:52:39 EST 2011 x86_64 > x86_64 x86_64 GNU/Linux > > cdr stop: > > 19:31:02 Maximum server connections 2 > 19:31:02 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked > 0, Plog used 61, Llog used 129 > > 19:33:17 CDR connection to server lost, id 23, name <g_ngdcops> > Reason: disconnect server > 19:33:17 CDR connection to server lost, id 22, name <g_ncdcops> > Reason: disconnect server > 19:33:17 CDR NIF Shutdown: connections all shutdown. > 19:33:17 CDR The NIF sub-component has shut down. > 19:33:21 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x16/0x(nil)) > > status=-1 SQLERR=0 thread 2893 queue control_send > 19:33:21 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x16/0x(nil)) > > status=-1 SQLERR=0 thread 2893 queue control_send > 19:33:21 CDR Queuer: rqmQueueTxn (21/0/0x16/0x0) failed on CNTRL > 19:33:21 CDR GC: operation control message construction failed (error -1). > 19:33:26 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x17/0x(nil)) > > status=-1 SQLERR=0 thread 2894 queue control_send > 19:33:26 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x17/0x(nil)) > > status=-1 SQLERR=0 thread 2894 queue control_send > 19:33:26 CDR Queuer: rqmQueueTxn (21/0/0x17/0x0) failed on CNTRL > 19:33:26 CDR GC: operation control message construction failed (error -1). > 19:33:28 DDR Log Snooping - Shutdown > 19:33:29 CDR shutdown complete > > cdr start: > > 19:35:01 CDR GC: catalog recovery begin > 19:35:01 CDR GC: operation control message construction failed (error -1). > 19:35:04 CDR GC: catalog recovery complete > 19:35:04 CDR queuer initialization complete > 19:35:04 CDR NIF listening on asf://g_nsofops > 19:35:04 DDR Log Snooping - Snooping started in log 9 > 19:35:04 CDR NIF site: 21 <g_ngdcops> clocks differ by 11 seconds > 19:35:04 Warning - The operating system time of day clock at the > > remote site differs from the local site. This can > > slow down the data-sync apply performance at the > > target server. This happens only if timestamp or > > stored procedure conflict resolution rules are > > being used. > 19:35:04 CDR: Re-connected to server, id 23, name <g_ngdcops> > 19:35:04 CDR NIF site: 21 <g_ncdcops> clocks differ by 11 seconds > 19:35:04 Warning - The operating system time of day clock at the > > remote site differs from the local site. This can > > slow down the data-sync apply performance at the > > target server. This happens only if timestamp or > > stored procedure conflict resolution rules are > > being used. > 19:35:04 CDR: Re-connected to server, id 22, name <g_ncdcops> > > --bcaec53961e2c1f5f304a25028c1 > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --bcaec52155857e5a3d04a251868c
Not sure what the question you are asking is. The messages look like normal stuff you see when you stop start replication. Also I wouldnt ignore time stamp diffrence if using -conflict=timestamp in production system as we do.
Sorry Karl, for any confuse. Build Version: 11.50.FC8 Build Number: N198 Build Host: vidar Build OS: Linux 2.6.9-34.ELsmp Build Date: Fri Nov 12 21:58:33 CST 2010 GLS Version: glslib-4.50.FC11 I wish the info now is more clearer. The following is log info generated by a cdr start and cdr stop. You can see two type of errors( see attached log info for more details): (1) 15:22:49 CDR GC: operation control message construction failed (error -1). (2) 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) status=-1 SQLERR=0 thread 225 queue control_send 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) status=-1 SQLERR=0 thread 225 queue control_send ..... Do you think they are normal? We did not have these errors in AIX for the past many years. ( IDS11.10) This happens recently after we moved to Linux/IDS11.50 FC8 Thanks, Frank 15:22:16 Checkpoint Completed: duration was 0 seconds. 15:22:16 Tue May 3 - loguniq 9, logpos 0x8fac018, timestamp: 0x1e28cf Interval: 1149 15:22:16 Maximum server connections 2 15:22:16 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked 0, Plog used 54, Llog used 142 15:22:47 CDR GC: catalog recovery begin 15:22:49 CDR GC: operation control message construction failed (error -1). 15:22:50 CDR GC: catalog recovery complete 15:22:50 CDR queuer initialization complete 15:22:50 CDR NIF listening on asf://g_nsofops 15:22:50 DDR Log Snooping - Snooping started in log 9 15:22:50 CDR: Re-connected to server, id 23, name <g_ngdcops> 15:22:51 CDR: Re-connected to server, id 22, name <g_ncdcops> 15:23:23 CDR connection to server lost, id 22, name <g_ncdcops> Reason: disconnect server 15:23:23 CDR connection to server lost, id 23, name <g_ngdcops> Reason: disconnect server 15:23:23 CDR NIF Shutdown: connections all shutdown. 15:23:23 CDR The NIF sub-component has shut down. 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) status=-1 SQLERR=0 thread 225 queue control_send 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) status=-1 SQLERR=0 thread 225 queue control_send 15:23:24 CDR Queuer: rqmQueueTxn (21/0/0x7/0x0) failed on CNTRL 15:23:24 CDR GC: operation control message construction failed (error -1). 15:23:29 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x8/0x(nil)) status=-1 SQLERR=0 thread 226 queue control_send 15:23:29 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x8/0x(nil)) status=-1 SQLERR=0 thread 226 queue control_send 15:23:29 CDR Queuer: rqmQueueTxn (21/0/0x8/0x0) failed on CNTRL 15:23:29 CDR GC: operation control message construction failed (error -1). 15:23:35 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x9/0x(nil)) status=-1 SQLERR=0 thread 227 queue control_send 15:23:35 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x9/0x(nil)) status=-1 SQLERR=0 thread 227 queue control_send 15:23:35 CDR Queuer: rqmQueueTxn (21/0/0x9/0x0) failed on CNTRL 15:23:35 CDR GC: operation control message construction failed (error -1). 15:23:36 DDR Log Snooping - Shutdown 15:23:37 CDR shutdown complete On Mon, May 2, 2011 at 10:00 PM, KARL OLIVER <karl.oliver@maf.govt.nz>wrote: > Not sure what the question you are asking is. > The messages look like normal stuff you see when you stop start > replication. > Also I wouldnt ignore time stamp diffrence if using > -conflict=timestamp in production system as we do. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --bcaec52155851d000604a260ee5e
Hello. It seems some trouble regarding your arch upgrade. Did you rebuild all your ER structure, "from zero"? How are your other engine versions? All were migrated to 11.50 also??? Here I had some troubles between 11.50 and 11.70, but rebuilding our ER from the latest version solved them all. Hope it clears you, anyway.... Best regards. Em 03/05/2011 11:38, FRANK escreveu: > Sorry Karl, for any confuse. > > Build Version: 11.50.FC8 > Build Number: N198 > Build Host: vidar > Build OS: Linux 2.6.9-34.ELsmp > Build Date: Fri Nov 12 21:58:33 CST 2010 > GLS Version: glslib-4.50.FC11 > > I wish the info now is more clearer. > > The following is log info generated by a cdr start and cdr stop. > > You can see two type of errors( see attached log info for more details): > > (1) > 15:22:49 CDR GC: operation control message construction failed (error -1). > > (2) > 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > ...... > > Do you think they are normal? > > We did not have these errors in AIX for the past many years. ( IDS11.10) > > This happens recently after we moved to Linux/IDS11.50 FC8 > > Thanks, > Frank > > 15:22:16 Checkpoint Completed: duration was 0 seconds. > 15:22:16 Tue May 3 - loguniq 9, logpos 0x8fac018, timestamp: 0x1e28cf > Interval: 1149 > > 15:22:16 Maximum server connections 2 > 15:22:16 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked > 0, Plog used 54, Llog used 142 > > 15:22:47 CDR GC: catalog recovery begin > 15:22:49 CDR GC: operation control message construction failed (error -1). > 15:22:50 CDR GC: catalog recovery complete > 15:22:50 CDR queuer initialization complete > 15:22:50 CDR NIF listening on asf://g_nsofops > 15:22:50 DDR Log Snooping - Snooping started in log 9 > 15:22:50 CDR: Re-connected to server, id 23, name<g_ngdcops> > 15:22:51 CDR: Re-connected to server, id 22, name<g_ncdcops> > > 15:23:23 CDR connection to server lost, id 22, name<g_ncdcops> > Reason: disconnect server > 15:23:23 CDR connection to server lost, id 23, name<g_ngdcops> > Reason: disconnect server > 15:23:23 CDR NIF Shutdown: connections all shutdown. > 15:23:23 CDR The NIF sub-component has shut down. > 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > 15:23:24 CDR Queuer: rqmQueueTxn (21/0/0x7/0x0) failed on CNTRL > 15:23:24 CDR GC: operation control message construction failed (error -1). > 15:23:29 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x8/0x(nil)) > > status=-1 SQLERR=0 thread 226 queue control_send > 15:23:29 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x8/0x(nil)) > > status=-1 SQLERR=0 thread 226 queue control_send > 15:23:29 CDR Queuer: rqmQueueTxn (21/0/0x8/0x0) failed on CNTRL > 15:23:29 CDR GC: operation control message construction failed (error -1). > 15:23:35 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x9/0x(nil)) > > status=-1 SQLERR=0 thread 227 queue control_send > 15:23:35 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x9/0x(nil)) > > status=-1 SQLERR=0 thread 227 queue control_send > 15:23:35 CDR Queuer: rqmQueueTxn (21/0/0x9/0x0) failed on CNTRL > 15:23:35 CDR GC: operation control message construction failed (error -1). > 15:23:36 DDR Log Snooping - Shutdown > 15:23:37 CDR shutdown complete > > On Mon, May 2, 2011 at 10:00 PM, KARL OLIVER<karl.oliver@maf.govt.nz>wrote: > >> Not sure what the question you are asking is. >> The messages look like normal stuff you see when you stop start >> replication. >> Also I wouldnt ignore time stamp diffrence if using >> -conflict=timestamp in production system as we do. >> >> >> >> > ******************************************************************************* >> Forum Note: Use "Reply" to post a response in the discussion forum. >> >> > --bcaec52155851d000604a260ee5e > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Alexandre Marini Tecnologia da Informação - DBA msn: alexandre_marini@hotmail.com SEFAZ-MS / SGI-UGSR / Sistemas IBM-Informix Cert-Info-Mgmt_color <Cert-Info-Mgmt_color.jpg> IBM Certified System Administrator - Informix Dynamic Server V10 / V11 See you at <http://www.iiug.org/conf/2011/iiug/>;
Alex, It is a brand new build! and are testing the brand new ER setup .... No relations to AIX db at all. We just created the ER servers it with cdr define server... All are totally new! Thanks, Frank [informix@maggie ~]$ uname -a Linux maggie 2.6.18-238.5.1.el5 #1 SMP Mon Feb 21 05:52:39 EST 2011 x86_64 x86_64 x86_64 GNU/Linux On Tue, May 3, 2011 at 11:49 AM, Alexandre Marini <amarini@fazenda.ms.gov.br > wrote: > Hello. > It seems some trouble regarding your arch upgrade. > Did you rebuild all your ER structure, "from zero"? > How are your other engine versions? All were migrated to 11.50 also??? > > Here I had some troubles between 11.50 and 11.70, but rebuilding our ER > from the latest version solved them all. > > Hope it clears you, anyway.... > Best regards. > > Em 03/05/2011 11:38, FRANK escreveu: > > Sorry Karl, for any confuse. > > Build Version: 11.50.FC8 > Build Number: N198 > Build Host: vidar > Build OS: Linux 2.6.9-34.ELsmp > Build Date: Fri Nov 12 21:58:33 CST 2010 > GLS Version: glslib-4.50.FC11 > > I wish the info now is more clearer. > > The following is log info generated by a cdr start and cdr stop. > > You can see two type of errors( see attached log info for more details): > > (1) > 15:22:49 CDR GC: operation control message construction failed (error -1). > > (2) > 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > ...... > > Do you think they are normal? > > We did not have these errors in AIX for the past many years. ( IDS11.10) > > This happens recently after we moved to Linux/IDS11.50 FC8 > > Thanks, > Frank > > 15:22:16 Checkpoint Completed: duration was 0 seconds. > 15:22:16 Tue May 3 - loguniq 9, logpos 0x8fac018, timestamp: 0x1e28cf > Interval: 1149 > > 15:22:16 Maximum server connections 2 > 15:22:16 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked > 0, Plog used 54, Llog used 142 > > 15:22:47 CDR GC: catalog recovery begin > 15:22:49 CDR GC: operation control message construction failed (error -1). > 15:22:50 CDR GC: catalog recovery complete > 15:22:50 CDR queuer initialization complete > 15:22:50 CDR NIF listening on asf://g_nsofops > 15:22:50 DDR Log Snooping - Snooping started in log 9 > 15:22:50 CDR: Re-connected to server, id 23, name <g_ngdcops> > 15:22:51 CDR: Re-connected to server, id 22, name <g_ncdcops> > > 15:23:23 CDR connection to server lost, id 22, name <g_ncdcops> > Reason: disconnect server > 15:23:23 CDR connection to server lost, id 23, name <g_ngdcops> > Reason: disconnect server > 15:23:23 CDR NIF Shutdown: connections all shutdown. > 15:23:23 CDR The NIF sub-component has shut down. > 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) > > status=-1 SQLERR=0 thread 225 queue control_send > 15:23:24 CDR Queuer: rqmQueueTxn (21/0/0x7/0x0) failed on CNTRL > 15:23:24 CDR GC: operation control message construction failed (error -1). > 15:23:29 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x8/0x(nil)) > > status=-1 SQLERR=0 thread 226 queue control_send > 15:23:29 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x8/0x(nil)) > > status=-1 SQLERR=0 thread 226 queue control_send > 15:23:29 CDR Queuer: rqmQueueTxn (21/0/0x8/0x0) failed on CNTRL > 15:23:29 CDR GC: operation control message construction failed (error -1). > 15:23:35 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x9/0x(nil)) > > status=-1 SQLERR=0 thread 227 queue control_send > 15:23:35 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x9/0x(nil)) > > status=-1 SQLERR=0 thread 227 queue control_send > 15:23:35 CDR Queuer: rqmQueueTxn (21/0/0x9/0x0) failed on CNTRL > 15:23:35 CDR GC: operation control message construction failed (error -1). > 15:23:36 DDR Log Snooping - Shutdown > 15:23:37 CDR shutdown complete > > On Mon, May 2, 2011 at 10:00 PM, KARL OLIVER <karl.oliver@maf.govt.nz> <karl.oliver@maf.govt.nz>wrote: > > > Not sure what the question you are asking is. > The messages look like normal stuff you see when you stop start > replication. > Also I wouldnt ignore time stamp diffrence if using > -conflict=timestamp in production system as we do. > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > --bcaec52155851d000604a260ee5e > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > > Alexandre Marini > > Tecnologia da Informação - DBA > > msn: alexandre_marini@hotmail.com > > SEFAZ-MS / SGI-UGSR / Sistemas IBM-Informix > > [image: Cert-Info-Mgmt_color] <http://Cert-Info-Mgmt_color.jpg>; > > IBM Certified System Administrator - Informix Dynamic Server V10 / V11 > See you at <http://www.iiug.org/conf/2011/iiug/>; > > > --000e0cd11196697d2304a2613cfb
Ok, in that case I think you´d better open a PMR with your IBM support, since there is no sqlerr appearing, and probably no AF file is generated, right? If you could trace your sql error, it could be easier.... Best regards. Em 03/05/2011 12:00, FRANK escreveu: > Alex, > > It is a brand new build! and are testing the brand new ER setup .... No > relations to AIX db at all. > > We just created the ER servers it with cdr define server... All are > totally new! > > Thanks, > Frank > > [informix@maggie ~]$ uname -a > Linux maggie 2.6.18-238.5.1.el5 #1 SMP Mon Feb 21 05:52:39 EST 2011 x86_64 > x86_64 x86_64 GNU/Linux > > On Tue, May 3, 2011 at 11:49 AM, Alexandre Marini<amarini@fazenda.ms.gov.br >> wrote: >> Hello. >> It seems some trouble regarding your arch upgrade. >> Did you rebuild all your ER structure, "from zero"? >> How are your other engine versions? All were migrated to 11.50 also??? >> >> Here I had some troubles between 11.50 and 11.70, but rebuilding our ER >> from the latest version solved them all. >> >> Hope it clears you, anyway.... >> Best regards. >> >> Em 03/05/2011 11:38, FRANK escreveu: >> >> Sorry Karl, for any confuse. >> >> Build Version: 11.50.FC8 >> Build Number: N198 >> Build Host: vidar >> Build OS: Linux 2.6.9-34.ELsmp >> Build Date: Fri Nov 12 21:58:33 CST 2010 >> GLS Version: glslib-4.50.FC11 >> >> I wish the info now is more clearer. >> >> The following is log info generated by a cdr start and cdr stop. >> >> You can see two type of errors( see attached log info for more details): >> >> (1) >> 15:22:49 CDR GC: operation control message construction failed (error -1). >> >> (2) >> 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) >> >> status=-1 SQLERR=0 thread 225 queue control_send >> 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) >> >> status=-1 SQLERR=0 thread 225 queue control_send >> ...... >> >> Do you think they are normal? >> >> We did not have these errors in AIX for the past many years. ( IDS11.10) >> >> This happens recently after we moved to Linux/IDS11.50 FC8 >> >> Thanks, >> Frank >> >> 15:22:16 Checkpoint Completed: duration was 0 seconds. >> 15:22:16 Tue May 3 - loguniq 9, logpos 0x8fac018, timestamp: 0x1e28cf >> Interval: 1149 >> >> 15:22:16 Maximum server connections 2 >> 15:22:16 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns blocked >> 0, Plog used 54, Llog used 142 >> >> 15:22:47 CDR GC: catalog recovery begin >> 15:22:49 CDR GC: operation control message construction failed (error -1). >> 15:22:50 CDR GC: catalog recovery complete >> 15:22:50 CDR queuer initialization complete >> 15:22:50 CDR NIF listening on asf://g_nsofops >> 15:22:50 DDR Log Snooping - Snooping started in log 9 >> 15:22:50 CDR: Re-connected to server, id 23, name<g_ngdcops> >> 15:22:51 CDR: Re-connected to server, id 22, name<g_ncdcops> >> >> 15:23:23 CDR connection to server lost, id 22, name<g_ncdcops> >> Reason: disconnect server >> 15:23:23 CDR connection to server lost, id 23, name<g_ngdcops> >> Reason: disconnect server >> 15:23:23 CDR NIF Shutdown: connections all shutdown. >> 15:23:23 CDR The NIF sub-component has shut down. >> 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) >> >> status=-1 SQLERR=0 thread 225 queue control_send >> 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) >> >> status=-1 SQLERR=0 thread 225 queue control_send >> 15:23:24 CDR Queuer: rqmQueueTxn (21/0/0x7/0x0) failed on CNTRL >> 15:23:24 CDR GC: operation control message construction failed (error -1). >> 15:23:29 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x8/0x(nil)) >> >> status=-1 SQLERR=0 thread 226 queue control_send >> 15:23:29 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x8/0x(nil)) >> >> status=-1 SQLERR=0 thread 226 queue control_send >> 15:23:29 CDR Queuer: rqmQueueTxn (21/0/0x8/0x0) failed on CNTRL >> 15:23:29 CDR GC: operation control message construction failed (error -1). >> 15:23:35 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x9/0x(nil)) >> >> status=-1 SQLERR=0 thread 227 queue control_send >> 15:23:35 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x9/0x(nil)) >> >> status=-1 SQLERR=0 thread 227 queue control_send >> 15:23:35 CDR Queuer: rqmQueueTxn (21/0/0x9/0x0) failed on CNTRL >> 15:23:35 CDR GC: operation control message construction failed (error -1). >> 15:23:36 DDR Log Snooping - Shutdown >> 15:23:37 CDR shutdown complete >> >> On Mon, May 2, 2011 at 10:00 PM, KARL OLIVER<karl.oliver@maf.govt.nz> > <karl.oliver@maf.govt.nz>wrote: >> >> Not sure what the question you are asking is. >> The messages look like normal stuff you see when you stop start >> replication. >> Also I wouldnt ignore time stamp diffrence if using >> -conflict=timestamp in production system as we do. >> >> >> >> >> >> > ******************************************************************************* >> Forum Note: Use "Reply" to post a response in the discussion forum. >> >> >> >> --bcaec52155851d000604a260ee5e >> >> >> > ******************************************************************************* >> Forum Note: Use "Reply" to post a response in the discussion forum. >> >> >> >> >> >> -- >> >> Alexandre Marini >> >> Tecnologia da Informação - DBA >> >> msn: alexandre_marini@hotmail.com >> >> SEFAZ-MS / SGI-UGSR / Sistemas IBM-Informix >> >> [image: Cert-Info-Mgmt_color]<http://Cert-Info-Mgmt_color.jpg>; >> >> IBM Certified System Administrator - Informix Dynamic Server V10 / V11 >> See you at<http://www.iiug.org/conf/2011/iiug/>; >> >> >> > --000e0cd11196697d2304a2613cfb > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Alexandre Marini Tecnologia da Informação - DBA msn: alexandre_marini@hotmail.com SEFAZ-MS / SGI-UGSR / Sistemas IBM-Informix Cert-Info-Mgmt_color <Cert-Info-Mgmt_color.jpg> IBM Certified System Administrator - Informix Dynamic Server V10 / V11 See you at <http://www.iiug.org/conf/2011/iiug/>;
No AF files... It is just caused by cdr start and cdr stop. No data replicated yet. Thanks, Frank On Tue, May 3, 2011 at 12:31 PM, Alexandre Marini <amarini@fazenda.ms.gov.br > wrote: > Ok, in that case I think you´d better open a PMR with your IBM support, > since there is no sqlerr appearing, and probably no AF file is > generated, right? > > If you could trace your sql error, it could be easier.... > Best regards. > > Em 03/05/2011 12:00, FRANK escreveu: > > Alex, > > > > It is a brand new build! and are testing the brand new ER setup .... No > > relations to AIX db at all. > > > > We just created the ER servers it with cdr define server... All are > > totally new! > > > > Thanks, > > Frank > > > > [informix@maggie ~]$ uname -a > > Linux maggie 2.6.18-238.5.1.el5 #1 SMP Mon Feb 21 05:52:39 EST 2011 > x86_64 > > x86_64 x86_64 GNU/Linux > > > > On Tue, May 3, 2011 at 11:49 AM, Alexandre Marini< > amarini@fazenda.ms.gov.br > >> wrote: > >> Hello. > >> It seems some trouble regarding your arch upgrade. > >> Did you rebuild all your ER structure, "from zero"? > >> How are your other engine versions? All were migrated to 11.50 also??? > >> > >> Here I had some troubles between 11.50 and 11.70, but rebuilding our ER > >> from the latest version solved them all. > >> > >> Hope it clears you, anyway.... > >> Best regards. > >> > >> Em 03/05/2011 11:38, FRANK escreveu: > >> > >> Sorry Karl, for any confuse. > >> > >> Build Version: 11.50.FC8 > >> Build Number: N198 > >> Build Host: vidar > >> Build OS: Linux 2.6.9-34.ELsmp > >> Build Date: Fri Nov 12 21:58:33 CST 2010 > >> GLS Version: glslib-4.50.FC11 > >> > >> I wish the info now is more clearer. > >> > >> The following is log info generated by a cdr start and cdr stop. > >> > >> You can see two type of errors( see attached log info for more details): > >> > >> (1) > >> 15:22:49 CDR GC: operation control message construction failed (error > -1). > >> > >> (2) > >> 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 225 queue control_send > >> 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 225 queue control_send > >> ...... > >> > >> Do you think they are normal? > >> > >> We did not have these errors in AIX for the past many years. ( IDS11.10) > >> > >> This happens recently after we moved to Linux/IDS11.50 FC8 > >> > >> Thanks, > >> Frank > >> > >> 15:22:16 Checkpoint Completed: duration was 0 seconds. > >> 15:22:16 Tue May 3 - loguniq 9, logpos 0x8fac018, timestamp: 0x1e28cf > >> Interval: 1149 > >> > >> 15:22:16 Maximum server connections 2 > >> 15:22:16 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns > blocked > >> 0, Plog used 54, Llog used 142 > >> > >> 15:22:47 CDR GC: catalog recovery begin > >> 15:22:49 CDR GC: operation control message construction failed (error > -1). > >> 15:22:50 CDR GC: catalog recovery complete > >> 15:22:50 CDR queuer initialization complete > >> 15:22:50 CDR NIF listening on asf://g_nsofops > >> 15:22:50 DDR Log Snooping - Snooping started in log 9 > >> 15:22:50 CDR: Re-connected to server, id 23, name<g_ngdcops> > >> 15:22:51 CDR: Re-connected to server, id 22, name<g_ncdcops> > >> > >> 15:23:23 CDR connection to server lost, id 22, name<g_ncdcops> > >> Reason: disconnect server > >> 15:23:23 CDR connection to server lost, id 23, name<g_ngdcops> > >> Reason: disconnect server > >> 15:23:23 CDR NIF Shutdown: connections all shutdown. > >> 15:23:23 CDR The NIF sub-component has shut down. > >> 15:23:24 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x7/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 225 queue control_send > >> 15:23:24 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x7/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 225 queue control_send > >> 15:23:24 CDR Queuer: rqmQueueTxn (21/0/0x7/0x0) failed on CNTRL > >> 15:23:24 CDR GC: operation control message construction failed (error > -1). > >> 15:23:29 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x8/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 226 queue control_send > >> 15:23:29 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x8/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 226 queue control_send > >> 15:23:29 CDR Queuer: rqmQueueTxn (21/0/0x8/0x0) failed on CNTRL > >> 15:23:29 CDR GC: operation control message construction failed (error > -1). > >> 15:23:35 CDR RQM: rqmSpoolHeader() failed for KEY (21/0/0x0x9/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 227 queue control_send > >> 15:23:35 CDR RQM: rqmSpoolTxn() failed for KEY (21/0/0x0x9/0x(nil)) > >> > >> status=-1 SQLERR=0 thread 227 queue control_send > >> 15:23:35 CDR Queuer: rqmQueueTxn (21/0/0x9/0x0) failed on CNTRL > >> 15:23:35 CDR GC: operation control message construction failed (error > -1). > >> 15:23:36 DDR Log Snooping - Shutdown > >> 15:23:37 CDR shutdown complete > >> > >> On Mon, May 2, 2011 at 10:00 PM, KARL OLIVER<karl.oliver@maf.govt.nz> > > <karl.oliver@maf.govt.nz>wrote: > >> > >> Not sure what the question you are asking is. > >> The messages look like normal stuff you see when you stop start > >> replication. > >> Also I wouldnt ignore time stamp diffrence if using > >> -conflict=timestamp in production system as we do. > >> > >> > >> > >> > >> > >> > > > > ******************************************************************************* > >> Forum Note: Use "Reply" to post a response in the discussion forum. > >> > >> > >> > >> --bcaec52155851d000604a260ee5e > >> > >> > >> > > > > ******************************************************************************* > >> Forum Note: Use "Reply" to post a response in the discussion forum. > >> > >> > >> > >> > >> > >> -- > >> > >> Alexandre Marini > >> > >> Tecnologia da Informação - DBA > >> > >> msn: alexandre_marini@hotmail.com > >> > >> SEFAZ-MS / SGI-UGSR / Sistemas IBM-Informix > >> > >> [image: Cert-Info-Mgmt_color]<http://Cert-Info-Mgmt_color.jpg>; > >> > >> IBM Certified System Administrator - Informix Dynamic Server V10 / V11 > >> See you at<http://www.iiug.org/conf/2011/iiug/>; > >> > >> > >> > > --000e0cd11196697d2304a2613cfb > > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > > Alexandre Marini > > Tecnologia da Informação - DBA > > msn: alexandre_marini@hotmail.com > > SEFAZ-MS / SGI-UGSR / Sistemas IBM-Informix > > Cert-Info-Mgmt_color <Cert-Info-Mgmt_color.jpg> > > IBM Certified System Administrator - Informix Dynamic Server V10 / V11 > > See you at > > <http://www.iiug.org/conf/2011/iiug/>; > > > > *******************************************************************************@@
If I am reading it right cdr finderr 1 1 A connection does not yet exist for the given server. Is replication actually working ?
Replication is working. data replicated fine. On Tue, May 3, 2011 at 3:55 PM, KARL OLIVER <karl.oliver@maf.govt.nz> wrote: > If I am reading it right > cdr finderr 1 > 1 A connection does not yet exist for the given server. > Is replication actually working ? > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --000e0cd50a5ca36c7b04a265fb2d