RE: Enterprise Replication question

Any chance of some assistance with this? I've had a further look and I can't make the -e option work anywhere in my ER replicates, from starting new ones to changing existing ones or through the use of a replicate set. Is there something I'm missing here? Do I need to turn this feature on somewhere - or tell the instance to use it? I will note that there are a number of other transaction based replicates running between the two servers and a heap of other servers also sending data back to the hades server (destination). ER doesn't just run all replicates at the fastest of any of those defined does it? Jarrod -----Original Message----- From: informix-list-bounces@iiug.org [mailto:informix-list-bounces@iiug.org] On Behalf Of Jarrod Teale Sent: Monday, 30 July 2007 4:14 p.m. To: informix-list@iiug.org Subject: RE: Enterprise Replication question Madison, I defined the replicate like this > cdr define repl -C ignore sirrp71_stan_tagsamples \\ > "R stan@g_hades:sdev.tagsamples_71" "SELECT * FROM tagsamples_71" \\ > "P stan@g_sirrp71:sdev.tagsamples" "SELECT * FROM tagsamples" Made this adjustment > cdr modify repl --ignoredel y sirrp71_stan_tagsamples Then made the time adjustment > cdr modify repl -e 10 sirrp71_stan_tagsamples I've also tried deleting the replicate and recreating with the -e flag in the define step but still see transactions rolling in every few seconds Thanks Jarrod -----Original Message----- From: Madison Pruet [mailto:mpruet1@verizon.net] Sent: Monday, 30 July 2007 4:09 p.m. To: Jarrod Teale Cc: informix-list@iiug.org Subject: Re: Enterprise Replication question Jarrod Teale wrote: > Hi, > Running 10.00.UC3 -> 10.00.UC3 replication, both on RedHat EL 4 > > I set up a one-way replicate to transfer data and it works beautifully > (Go Informix) but I forgot to set the frequency option in the > replicate. Please post the command you used to define the replicate and also the 'cdr modify repl' command that you issued. M.P. > > I've now set it to the right thing, using cdr modify replicate but it > doesn't seem to have taken effect. (You can see the 10 minute > frequency > below) > > The records are still arriving at the central server at the end of > each transaction, not caching for 10 minutes before being sent > > Any ideas? > > Jarrod Teale > Fonterra New Zealand. > > > > [informix@sirrp71(10.71.64.130) ~]$ cdr list repl > sirrp71_stan_tagsamples > > DEFINED REPLICATES ATTRIBUTES > ------------------------------ > REPLICATE: sirrp71_stan_tagsamples > STATE: Active ON:sirrp71 > CONFLICT: Ignore > FREQUENCY: every 00:10 > QUEUE SIZE: 0 > PARTICIPANT: stan:sdev.tagsamples > OPTIONS: transaction,fullrow > > [informix@sirrp71(10.71.64.130) ~]$ cdr list repl brief > sirrp71_stan_tagsamples > > REPLICATE TABLE SELECT > ---------------------------------------------------------------------- > ------ sirrp71_stan_tagsamples stan@g_sirrp71:sdev.tagsamples > <mailto:stan@g_sirrp71:sdev.tagsamples> SELECT * FROM tagsamples > sirrp71_stan_tagsamples db@g_hades:owner.table > <mailto:db@g_hades:owner.table> select * from table > > DISCLAIMER: > This email contains confidential information and may be legally > privileged. If you are not the intended recipient or have received > this email in error, please notify the sender immediately and destroy this email. > You may not use, disclose or copy this email or its attachments in any way. > Any opinions expressed in this email are those of the author and are > not necessarily those of the Fonterra Co-operative Group. > http://www.fonterra.com/ > DISCLAIMER: This email contains confidential information and may be legally privileged. If you are not the intended recipient or have received this email in error, please notify the sender immediately and destroy this email. You may not use, disclose or copy this email or its attachments in any way. Any opinions expressed in this email are those of the author and are not necessarily those of the Fonterra Co-operative Group. http://www.fonterra.com/ _______________________________________________ Informix-list mailing list Informix-list@iiug.org http://www.iiug.org/mailman/listinfo/informix-list DISCLAIMER: This email contains confidential information and may be legally privileged. If you are not the intended recipient or have received this email in error, please notify the sender immediately and destroy this email. You may not use, disclose or copy this email or its attachments in any way. Any opinions expressed in this email are those of the author and are not necessarily those of the Fonterra Co-operative Group. http://www.fonterra.com/

Jarrod Teale wrote: > Any chance of some assistance with this? I've had a further look and I > can't make the -e option work anywhere in my ER replicates, from > starting new ones to changing existing ones or through the use of a > replicate set. > > Is there something I'm missing here? Do I need to turn this feature on > somewhere - or tell the instance to use it? > > I will note that there are a number of other transaction based > replicates running between the two servers and a heap of other servers > also sending data back to the hades server (destination). ER doesn't > just run all replicates at the fastest of any of those defined does it? I just tested this and it seemed to be working ok. What is the output of onstat -g rep on the source system. (With timed based replication, it's the source system that manages the transmission of the changes.) Just as an aside, what is the reason that you are wanting to use the -e 10 option? > > Jarrod > > -----Original Message----- > From: informix-list-bounces@iiug.org > [mailto:informix-list-bounces@iiug.org] On Behalf Of Jarrod Teale > Sent: Monday, 30 July 2007 4:14 p.m. > To: informix-list@iiug.org > Subject: RE: Enterprise Replication question > > Madison, > > I defined the replicate like this > >> cdr define repl -C ignore sirrp71_stan_tagsamples \\ >> "R stan@g_hades:sdev.tagsamples_71" "SELECT * FROM > tagsamples_71" \\ >> "P stan@g_sirrp71:sdev.tagsamples" "SELECT * FROM tagsamples" > > Made this adjustment > >> cdr modify repl --ignoredel y sirrp71_stan_tagsamples > > Then made the time adjustment > >> cdr modify repl -e 10 sirrp71_stan_tagsamples > > I've also tried deleting the replicate and recreating with the -e flag > in the define step but still see transactions rolling in every few > seconds > > Thanks > > Jarrod > > -----Original Message----- > From: Madison Pruet [mailto:mpruet1@verizon.net] > Sent: Monday, 30 July 2007 4:09 p.m. > To: Jarrod Teale > Cc: informix-list@iiug.org > Subject: Re: Enterprise Replication question > > Jarrod Teale wrote: >> Hi, >> Running 10.00.UC3 -> 10.00.UC3 replication, both on RedHat EL 4 >> >> I set up a one-way replicate to transfer data and it works beautifully > >> (Go Informix) but I forgot to set the frequency option in the >> replicate. > > Please post the command you used to define the replicate and also the > 'cdr modify repl' command that you issued. > > M.P. > >> >> I've now set it to the right thing, using cdr modify replicate but it >> doesn't seem to have taken effect. (You can see the 10 minute >> frequency >> below) >> >> The records are still arriving at the central server at the end of >> each transaction, not caching for 10 minutes before being sent >> >> Any ideas? >> >> Jarrod Teale >> Fonterra New Zealand. >> >> >> >> [informix@sirrp71(10.71.64.130) ~]$ cdr list repl >> sirrp71_stan_tagsamples >> >> DEFINED REPLICATES ATTRIBUTES >> ------------------------------ >> REPLICATE: sirrp71_stan_tagsamples >> STATE: Active ON:sirrp71 >> CONFLICT: Ignore >> FREQUENCY: every 00:10 >> QUEUE SIZE: 0 >> PARTICIPANT: stan:sdev.tagsamples >> OPTIONS: transaction,fullrow >> >> [informix@sirrp71(10.71.64.130) ~]$ cdr list repl brief >> sirrp71_stan_tagsamples >> >> REPLICATE TABLE SELECT >> ---------------------------------------------------------------------- >> ------ sirrp71_stan_tagsamples stan@g_sirrp71:sdev.tagsamples >> <mailto:stan@g_sirrp71:sdev.tagsamples> SELECT * FROM > tagsamples >> sirrp71_stan_tagsamples db@g_hades:owner.table >> <mailto:db@g_hades:owner.table> select * from table >> >> DISCLAIMER: >> This email contains confidential information and may be legally >> privileged. If you are not the intended recipient or have received >> this email in error, please notify the sender immediately and destroy > this email. >> You may not use, disclose or copy this email or its attachments in any > way. >> Any opinions expressed in this email are those of the author and are >> not necessarily those of the Fonterra Co-operative Group. >> http://www.fonterra.com/ >> > > > > > DISCLAIMER: > This email contains confidential information and may be legally > privileged. If you are not the intended recipient or have received this > email in error, please notify the sender immediately and destroy this > email. > You may not use, disclose or copy this email or its attachments in any > way. > Any opinions expressed in this email are those of the author and are not > necessarily those of the Fonterra Co-operative Group. > http://www.fonterra.com/ > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list > > > DISCLAIMER: > This email contains confidential information and may be legally privileged. If you are not the intended recipient or have received this email in error, please notify the sender immediately and destroy this email. > You may not use, disclose or copy this email or its attachments in any way. > Any opinions expressed in this email are those of the author and are not necessarily those of the Fonterra Co-operative Group. > http://www.fonterra.com/ >

The output looks fine (I've set it up for five minutes) as below but the transactions are still coming back VERY fast (each second or two) I've included date stamp and current record data from the target table on the central server - as you can see it's pretty much immediate up to date (the servers are time synced.) I want to look at the -e 10 option to help with compression I have CDR_NIFCOMPRESS set to 9 and I want to see what difference in network traffic occurs when I batch the transactions I operate in a constrained network environment where I don't have full control of my WAN links - The less I use them the less I get yelled at. If I batch the data into 3-10 minute blocks, I'm hoping it will bring my daily volume of data transferred between servers down. I can happily live with the data being this old in the central system(it updates on the source system every few seconds) [informix@sirrp71(10.71.64.130) ~]$ onstat -g rep IBM Informix Dynamic Server Version 10.00.UC3 -- On-Line -- Up 14 days 03:18:50 -- 144028 Kbytes Time based replicates Name Frequency Last Next ======================================================================= stan_set Every 5 min Aug 2 14:38:11 Aug 2 14:43:11 Replset sirrp71_stan_tags Every 5 min Aug 2 14:38:11 Aug 2 14:43:11 sirrp71_stan_tagsa Every 5 min Aug 2 14:38:11 Aug 2 14:43:11 sirrp71_apcapp_cyp Every 30 min Aug 2 14:30:38 Aug 2 15:00:38 apcappstatus_set Every 30 min Aug 2 14:30:38 Aug 2 15:00:38 Replset Schedule manager Cb: 0x13fd6e18 State: 0x8100 <CDRINIT,CDRRUNNING> Event Thread When ============================================ CDRDS CDREvent 00:00:04 CDRRepSched CDRRep 00:04:37 CDRRepSched CDRRep 00:04:37 CDRRepSched CDRRep 00:04:37 CDRRepSched CDRRep 00:22:04 CDRRepSched CDRRep 00:22:04 [sdev@hades ~]$ date Thu Aug 2 14:40:41 NZST 2007 [sdev@hades ~]$ dbaccess stan <<! > select max(sample_dt) from tagsamples_71 > ! Database selected. (max) 2007-08-02 14:40:42.000 1 row(s) retrieved. Database closed. -----Original Message----- From: Madison Pruet [mailto:mpruet1@verizon.net] Sent: Thursday, 2 August 2007 2:32 p.m. To: Jarrod Teale Cc: informix-list@iiug.org Subject: Re: Enterprise Replication question Jarrod Teale wrote: > Any chance of some assistance with this? I've had a further look and I > can't make the -e option work anywhere in my ER replicates, from > starting new ones to changing existing ones or through the use of a > replicate set. > > Is there something I'm missing here? Do I need to turn this feature on > somewhere - or tell the instance to use it? > > I will note that there are a number of other transaction based > replicates running between the two servers and a heap of other servers > also sending data back to the hades server (destination). ER doesn't > just run all replicates at the fastest of any of those defined does it? I just tested this and it seemed to be working ok. What is the output of onstat -g rep on the source system. (With timed based replication, it's the source system that manages the transmission of the changes.) Just as an aside, what is the reason that you are wanting to use the -e 10 option? > > Jarrod > > -----Original Message----- > From: informix-list-bounces@iiug.org > [mailto:informix-list-bounces@iiug.org] On Behalf Of Jarrod Teale > Sent: Monday, 30 July 2007 4:14 p.m. > To: informix-list@iiug.org > Subject: RE: Enterprise Replication question > > Madison, > > I defined the replicate like this > >> cdr define repl -C ignore sirrp71_stan_tagsamples \\ >> "R stan@g_hades:sdev.tagsamples_71" "SELECT * FROM > tagsamples_71" \\ >> "P stan@g_sirrp71:sdev.tagsamples" "SELECT * FROM tagsamples" > > Made this adjustment > >> cdr modify repl --ignoredel y sirrp71_stan_tagsamples > > Then made the time adjustment > >> cdr modify repl -e 10 sirrp71_stan_tagsamples > > I've also tried deleting the replicate and recreating with the -e flag > in the define step but still see transactions rolling in every few > seconds > > Thanks > > Jarrod > > -----Original Message----- > From: Madison Pruet [mailto:mpruet1@verizon.net] > Sent: Monday, 30 July 2007 4:09 p.m. > To: Jarrod Teale > Cc: informix-list@iiug.org > Subject: Re: Enterprise Replication question > > Jarrod Teale wrote: >> Hi, >> Running 10.00.UC3 -> 10.00.UC3 replication, both on RedHat EL 4 >> >> I set up a one-way replicate to transfer data and it works beautifully > >> (Go Informix) but I forgot to set the frequency option in the >> replicate. > > Please post the command you used to define the replicate and also the > 'cdr modify repl' command that you issued. > > M.P. > >> >> I've now set it to the right thing, using cdr modify replicate but it >> doesn't seem to have taken effect. (You can see the 10 minute >> frequency >> below) >> >> The records are still arriving at the central server at the end of >> each transaction, not caching for 10 minutes before being sent >> >> Any ideas? >> >> Jarrod Teale >> Fonterra New Zealand. >> >> >> >> [informix@sirrp71(10.71.64.130) ~]$ cdr list repl >> sirrp71_stan_tagsamples >> >> DEFINED REPLICATES ATTRIBUTES >> ------------------------------ >> REPLICATE: sirrp71_stan_tagsamples >> STATE: Active ON:sirrp71 >> CONFLICT: Ignore >> FREQUENCY: every 00:10 >> QUEUE SIZE: 0 >> PARTICIPANT: stan:sdev.tagsamples >> OPTIONS: transaction,fullrow >> >> [informix@sirrp71(10.71.64.130) ~]$ cdr list repl brief >> sirrp71_stan_tagsamples >> >> REPLICATE TABLE SELECT >> ---------------------------------------------------------------------- >> ------ sirrp71_stan_tagsamples stan@g_sirrp71:sdev.tagsamples >> <mailto:stan@g_sirrp71:sdev.tagsamples> SELECT * FROM > tagsamples >> sirrp71_stan_tagsamples db@g_hades:owner.table >> <mailto:db@g_hades:owner.table> select * from table >> >> DISCLAIMER: >> This email contains confidential information and may be legally >> privileged. If you are not the intended recipient or have received >> this email in error, please notify the sender immediately and destroy > this email. >> You may not use, disclose or copy this email or its attachments in any > way. >> Any opinions expressed in this email are those of the author and are >> not necessarily those of the Fonterra Co-operative Group. >> http://www.fonterra.com/ >> > > > > > DISCLAIMER: > This email contains confidential information and may be legally > privileged. If you are not the intended recipient or have received this > email in error, please notify the sender i

Jarrod Teale wrote: > The output looks fine (I've set it up for five minutes) as below but the > transactions are still coming back VERY fast (each second or two) > I've included date stamp and current record data from the target table > on the central server - as you can see it's pretty much immediate up to > date (the servers are time synced.) I wouldn't do timed based replication. I put in a whole bunch of warning messages when it is used. ;-)... The reason is that when using timed based replication, the transaction is split into multiple transmission units and you simply can't support things such as referential integrity when doing that. A possible solution for you which would allow you to maintain the transactional unit - and to control the transmissions would be to use "cdr disconnect" and "cdr connect" periodically. This would cause the network connection to be dropped between the servers and then later reestablished. After running cdr connect, you could wait for a while then issue a disconnect. As an example, suppose that you had the following deamon... while [ 1 ];do cdr connect -c <source> <target> sleep 60 cdr disconnect -c <source> <target> sleep 600 done This would allow traffic for a min., then would disconnect and remain disconnected for 10 min. Then reconnect, etc.... M.P. > > I want to look at the -e 10 option to help with compression > I have CDR_NIFCOMPRESS set to 9 and I want to see what difference in > network traffic occurs when I batch the transactions > I operate in a constrained network environment where I don't have full > control of my WAN links - The less I use them the less I get yelled at. > If I batch the data into 3-10 minute blocks, I'm hoping it will bring my > daily volume of data transferred between servers down. > > I can happily live with the data being this old in the central system(it > updates on the source system every few seconds) > > [informix@sirrp71(10.71.64.130) ~]$ onstat -g rep > > IBM Informix Dynamic Server Version 10.00.UC3 -- On-Line -- Up 14 > days 03:18:50 -- 144028 Kbytes > > Time based replicates> > Name Frequency Last Next > ======================================================================= > stan_set Every 5 min Aug 2 14:38:11 Aug 2 14:43:11 > Replset > sirrp71_stan_tags Every 5 min Aug 2 14:38:11 Aug 2 14:43:11 > sirrp71_stan_tagsa Every 5 min Aug 2 14:38:11 Aug 2 14:43:11 > sirrp71_apcapp_cyp Every 30 min Aug 2 14:30:38 Aug 2 15:00:38 > apcappstatus_set Every 30 min Aug 2 14:30:38 Aug 2 15:00:38 > Replset > > Schedule manager Cb: 0x13fd6e18 State: 0x8100 <CDRINIT,CDRRUNNING> > > Event Thread When > ============================================ > CDRDS CDREvent 00:00:04 > CDRRepSched CDRRep 00:04:37 > CDRRepSched CDRRep 00:04:37 > CDRRepSched CDRRep 00:04:37 > CDRRepSched CDRRep 00:22:04 > CDRRepSched CDRRep 00:22:04 > > > [sdev@hades ~]$ date > Thu Aug 2 14:40:41 NZST 2007 > [sdev@hades ~]$ dbaccess stan <<! >> select max(sample_dt) from tagsamples_71 >> ! > > Database selected. > > (max) > 2007-08-02 14:40:42.000 > 1 row(s) retrieved. > > Database closed. > > > -----Original Message----- > From: Madison Pruet [mailto:mpruet1@verizon.net] > Sent: Thursday, 2 August 2007 2:32 p.m. > To: Jarrod Teale > Cc: informix-list@iiug.org > Subject: Re: Enterprise Replication question > > Jarrod Teale wrote: >> Any chance of some assistance with this? I've had a further look and I > >> can't make the -e option work anywhere in my ER replicates, from >> starting new ones to changing existing ones or through the use of a >> replicate set. >> >> Is there something I'm missing here? Do I need to turn this feature on > >> somewhere - or tell the instance to use it? >> >> I will note that there are a number of other transaction based >> replicates running between the two servers and a heap of other servers > >> also sending data back to the hades server (destination). ER doesn't >> just run all replicates at the fastest of any of those defined does > it? > > I just tested this and it seemed to be working ok. What is the output > of onstat -g rep on the source system. (With timed based replication, > it's the source system that manages the transmission of the changes.) > > Just as an aside, what is the reason that you are wanting to use the -e > 10 option? > > >> Jarrod >> >> -----Original Message----- >> From: informix-list-bounces@iiug.org >> [mailto:informix-list-bounces@iiug.org] On Behalf Of Jarrod Teale >> Sent: Monday, 30 July 2007 4:14 p.m. >> To: informix-list@iiug.org >> Subject: RE: Enterprise Replication question >> >> Madison, >> >> I defined the replicate like this >> >>> cdr define repl -C ignore sirrp71_stan_tagsamples \\ >>> "R stan@g_hades:sdev.tagsamples_71" "SELECT * FROM >> tagsamples_71" \\ >>> "P stan@g_sirrp71:sdev.tagsamples" "SELECT * FROM tagsamples" >> Made this adjustment >> >>> cdr modify repl --ignoredel y sirrp71_stan_tagsamples >> Then made the time adjustment >> >>> cdr modify repl -e 10 sirrp71_stan_tagsamples >> I've also tried deleting the replicate and recreating with the -e flag >> in the define step but still see transactions rolling in every few >> seconds >> >> Thanks >> >> Jarrod >> >> -----Original Message----- >> From: Madison Pruet [mailto:mpruet1@verizon.net] >> Sent: Monday, 30 July 2007 4:09 p.m. >> To: Jarrod Teale >> Cc: informix-list@iiug.org >> Subject: Re: Enterprise Replication question >> >> Jarrod Teale wrote: >>> Hi, >>> Running 10.00.UC3 -> 10.00.UC3 replication, both on RedHat EL 4 >>> >>> I set up a one-way replicate to transfer data and it works > beautifully >>> (Go Informix) but I forgot to set the frequency option in the >>> replicate. >> Please post the command you used to define the replicate and also the >> 'cdr modify repl' command that you issued. >> >> M.P. >> >>> >>> I've now set it to the right thing, using cdr modify replicate but it > >>> doesn't seem to have taken effect. (You can see the 10 minute >>> frequency >>> below) >>> >>> The records are still arriving at the central server at the end of >>> each transaction, not caching for 10 minutes before being sent >>> >>> Any ideas? >>> >>> Jarrod Teale >>> Fonterra New Zealand. >>> >>> >>> >>> [informix@sirrp71(10.71.64.130) ~]$ cdr list repl >>> sirrp71_stan_tagsamples >>> >>> DEFINED REPLICATES ATTRIBUTES >>> ------------------------------ >>> REPLICATE: sirrp71_stan_tagsamples >>> STATE: Active ON:sirrp71 >>> CONFLICT: Ignore >>> FREQUENCY: every 00:10 >>> QUEUE SIZE: 0 >>> PARTICIPANT: stan:sdev.tagsamples >>> OPTIONS: transacti