Re: Informix Data Replication
You might want to specify wheither you are considering HDR or ER (Enterprise Replication). Both use the logs to apply data on the target. HDR simply puts the target instance into recovery mode and ships the logs from the source to the target. ER rebuilds the changed rows from the log files and selectively applies them to various targets. wesslim@my-deja.com wrote: > Has anyone used Informix Data Replication extensively? I work for a > corporation who is considering using this Informix product and would like to > find out specifically, how agent uses the logs to replicate and also and > overhead or speed issues related to it's use. > > thanks > > Sent via Deja.com http://www.deja.com/ > Before you buy.
Madison, Thanks for the tip. We are considering Enterprise Replication. We are a 24 x 7 company and require our database to be online at all times. We have invested heavily in a fail-over scenario that will allow the database to be up during hardware/electrical failures, but have not found a solution to upgrade/scheduled downtime scenarios. Currently we have one standard database, but are considering replicating another database that can be used during our upgrade periods (about 1 a month). Data replication seems to be the answer, as we could move clients to the secondary database and replicate their work back to the primary when we are finished with the update. However, we are concerned about overhead (i.e. performance impacts) and the specifics of the logging and agents that run for data replication. In article <3A22EA05.E78EE5B6@home.com>, Madison Pruet <mpruet@home.com> wrote: > You might want to specify wheither you are considering HDR or ER (Enterprise > Replication). Both use the logs to apply data on the target. HDR simply puts > the target instance into recovery mode and ships the logs from the source to the > target. ER rebuilds the changed rows from the log files and selectively applies > them to various targets. > > wesslim@my-deja.com wrote: > > > Has anyone used Informix Data Replication extensively? I work for a > > corporation who is considering using this Informix product and would like to > > find out specifically, how agent uses the logs to replicate and also and > > overhead or speed issues related to it's use. > > > > thanks > > > > Sent via Deja.com http://www.deja.com/ > > Before you buy. > > Sent via Deja.com http://www.deja.com/ Before you buy.
If you want to be able to upgrade one of the servers in a replicated group of servers, then ER is the way to go. The stragety that you have described is fairly easy to implement. Now for the overhead. --- I really hate these types of questions. They are always answered by "It depends...". The overhead of ER varies according to average transaction size. We try to keep the queue in memory so that we can avoid any extra disk activity. If the average transaction is fairly small, then it is unlikly that spooling of the queue to disk will occur. That being the case, the cost of replications (i.e. log snooping/queueing/transmission to the target/etc..) is fairly small - maybe arround 5% of the original transaction. If the average transaction is larger such that we have to spool the queue to disk, then the cost of replication would be about 15% of the original transaction. So naturally, the smaller the average transaction size, the less overall impact. Also, the smaller the average transaction size, the lower the latency. One thing to think about however. If the transaction does deletes, then the replication threads will have to be putting those deleted rows into the deleted row tables. Therefor if there are a lot of deletes, then the cost of replication does increase somewhat. wesslim@my-deja.com wrote: > Madison, > > Thanks for the tip. We are considering Enterprise Replication. We are > a 24 x 7 company and require our database to be online at all times. We > have invested heavily in a fail-over scenario that will allow the > database to be up during hardware/electrical failures, but have not > found a solution to upgrade/scheduled downtime scenarios. Currently we > have one standard database, but are considering replicating another > database that can be used during our upgrade periods (about 1 a month). > Data replication seems to be the answer, as we could move clients to > the secondary database and replicate their work back to the primary when > we are finished with the update. However, we are concerned about > overhead (i.e. performance impacts) and the specifics of the logging and > agents that run for data replication. > > In article <3A22EA05.E78EE5B6@home.com>, > Madison Pruet <mpruet@home.com> wrote: > > You might want to specify wheither you are considering HDR or ER > (Enterprise > > Replication). Both use the logs to apply data on the target. HDR > simply puts > > the target instance into recovery mode and ships the logs from the > source to the > > target. ER rebuilds the changed rows from the log files and > selectively applies > > them to various targets. > > > > wesslim@my-deja.com wrote: > > > > > Has anyone used Informix Data Replication extensively? I work for a > > > corporation who is considering using this Informix product and would > like to > > > find out specifically, how agent uses the logs to replicate and also > and > > > overhead or speed issues related to it's use. > > > > > > thanks > > > > > > Sent via Deja.com http://www.deja.com/ > > > Before you buy. > > > > > > Sent via Deja.com http://www.deja.com/ > Before you buy.
Thanks for the info! I have one more question. Since the logging covers all transactions, what happens when you make a structural change? For example, while upgrading the primary server a change is made to index structure (drop and recreate). Once finished, you replicate all transactions that occurred back. Do the secondary and primary have to match structurally? This is with the assumption that replication is turned off during the upgrade and it is only used during downtimes to replicate one-way from the secondary to the primary. I would assume that the best scenario would be to upgrade the secondary first making all index changes before the failover and then upgrade the primary and then turn on data replication to pick up any transactions that occurred during the upgrade of the primary. Does this scenario make sense? I appreciate your input. Mashon Wessling Lead Technical Systems Specialist Domino's Pizza LLC In article <3A24513F.77876F1C@home.com>, Madison Pruet <mpruet@home.com> wrote: > If you want to be able to upgrade one of the servers in a replicated group > of servers, then ER is the way to go. The stragety that you have described > is fairly easy to implement. > > Now for the overhead. --- > > I really hate these types of questions. They are always answered by "It > depends...". > > The overhead of ER varies according to average transaction size. We try to > keep the queue in memory so that we can avoid any extra disk activity. If > the average transaction is fairly small, then it is unlikly that spooling of > the queue to disk will occur. That being the case, the cost of replications > (i.e. log snooping/queueing/transmission to the target/etc..) is fairly > small - maybe arround 5% of the original transaction. If the average > transaction is larger such that we have to spool the queue to disk, then the > cost of replication would be about 15% of the original transaction. So > naturally, the smaller the average transaction size, the less overall > impact. Also, the smaller the average transaction size, the lower the > latency. > > One thing to think about however. If the transaction does deletes, then the > replication threads will have to be putting those deleted rows into the > deleted row tables. Therefor if there are a lot of deletes, then the cost > of replication does increase somewhat. > > wesslim@my-deja.com wrote: > > > Madison, > > > > Thanks for the tip. We are considering Enterprise Replication. We are > > a 24 x 7 company and require our database to be online at all times. We > > have invested heavily in a fail-over scenario that will allow the > > database to be up during hardware/electrical failures, but have not > > found a solution to upgrade/scheduled downtime scenarios. Currently we > > have one standard database, but are considering replicating another > > database that can be used during our upgrade periods (about 1 a month). > > Data replication seems to be the answer, as we could move clients to > > the secondary database and replicate their work back to the primary when > > we are finished with the update. However, we are concerned about > > overhead (i.e. performance impacts) and the specifics of the logging and > > agents that run for data replication. > > > > In article <3A22EA05.E78EE5B6@home.com>, > > Madison Pruet <mpruet@home.com> wrote: > > > You might want to specify wheither you are considering HDR or ER > > (Enterprise > > > Replication). Both use the logs to apply data on the target. HDR > > simply puts > > > the target instance into recovery mode and ships the logs from the > > source to the > > > target. ER rebuilds the changed rows from the log files and > > selectively applies > > > them to various targets. > > > > > > wesslim@my-deja.com wrote: > > > > > > > Has anyone used Informix Data Replication extensively? I work for a > > > > corporation who is considering using this Informix product and would > > like to > > > > find out specifically, how agent uses the logs to replicate and also > > and > > > > overhead or speed issues related to it's use. > > > > > > > > thanks > > > > > > > > Sent via Deja.com http://www.deja.com/ > > > > Before you buy. > > > > > > > > > > Sent via Deja.com http://www.deja.com/ > > Before you buy. > > Sent via Deja.com http://www.deja.com/ Before you buy.
Only the data is replicated. An index change would not. The only index that would have to exist on both the source and the target would be the one generated by the primary key. wesslim@my-deja.com wrote: > Thanks for the info! > > I have one more question. Since the logging covers all transactions, > what happens when you make a structural change? > > For example, while upgrading the primary server a change is made to > index structure (drop and recreate). Once finished, you replicate all > transactions that occurred back. Do the secondary and primary have to > match structurally? This is with the assumption that replication is > turned off during the upgrade and it is only used during downtimes to > replicate one-way from the secondary to the primary. I would assume > that the best scenario would be to upgrade the secondary first making > all index changes before the failover and then upgrade the primary and > then turn on data replication to pick up any transactions that occurred > during the upgrade of the primary. Does this scenario make sense? > > I appreciate your input. > > Mashon Wessling > Lead Technical Systems Specialist > Domino's Pizza LLC > > In article <3A24513F.77876F1C@home.com>, > Madison Pruet <mpruet@home.com> wrote: > > If you want to be able to upgrade one of the servers in a replicated > group > > of servers, then ER is the way to go. The stragety that you have > described > > is fairly easy to implement. > > > > Now for the overhead. --- > > > > I really hate these types of questions. They are always answered by > "It > > depends...". > > > > The overhead of ER varies according to average transaction size. We > try to > > keep the queue in memory so that we can avoid any extra disk activity. > If > > the average transaction is fairly small, then it is unlikly that > spooling of > > the queue to disk will occur. That being the case, the cost of > replications > > (i.e. log snooping/queueing/transmission to the target/etc..) is > fairly > > small - maybe arround 5% of the original transaction. If the average > > transaction is larger such that we have to spool the queue to disk, > then the > > cost of replication would be about 15% of the original transaction. > So > > naturally, the smaller the average transaction size, the less overall > > impact. Also, the smaller the average transaction size, the lower the > > latency. > > > > One thing to think about however. If the transaction does deletes, > then the > > replication threads will have to be putting those deleted rows into > the > > deleted row tables. Therefor if there are a lot of deletes, then the > cost > > of replication does increase somewhat. > > > > wesslim@my-deja.com wrote: > > > > > Madison, > > > > > > Thanks for the tip. We are considering Enterprise Replication. We > are > > > a 24 x 7 company and require our database to be online at all times. > We > > > have invested heavily in a fail-over scenario that will allow the > > > database to be up during hardware/electrical failures, but have not > > > found a solution to upgrade/scheduled downtime scenarios. Currently > we > > > have one standard database, but are considering replicating another > > > database that can be used during our upgrade periods (about 1 a > month). > > > Data replication seems to be the answer, as we could move clients > to > > > the secondary database and replicate their work back to the primary > when > > > we are finished with the update. However, we are concerned about > > > overhead (i.e. performance impacts) and the specifics of the logging > and > > > agents that run for data replication. > > > > > > In article <3A22EA05.E78EE5B6@home.com>, > > > Madison Pruet <mpruet@home.com> wrote: > > > > You might want to specify wheither you are considering HDR or ER > > > (Enterprise > > > > Replication). Both use the logs to apply data on the target. HDR > > > simply puts > > > > the target instance into recovery mode and ships the logs from the > > > source to the > > > > target. ER rebuilds the changed rows from the log files and > > > selectively applies > > > > them to various targets. > > > > > > > > wesslim@my-deja.com wrote: > > > > > > > > > Has anyone used Informix Data Replication extensively? I work > for a > > > > > corporation who is considering using this Informix product and > would > > > like to > > > > > find out specifically, how agent uses the logs to replicate and > also > > > and > > > > > overhead or speed issues related to it's use. > > > > > > > > > > thanks > > > > > > > > > > Sent via Deja.com http://www.deja.com/ > > > > > Before you buy. > > > > > > > > > > > > > > Sent via Deja.com http://www.deja.com/ > > > Before you buy. > > > > > > Sent via Deja.com http://www.deja.com/ > Before you buy.
HDR is also a possible candidate for your situation. It has a lower CPU overhead than ER (5-7% against 15-20% in our environment) and is quite simple to deploy and maintain. One major drawback to it is that only two servers can participate - a primary and a secondary. Another minor drawback is that the two servers must be identical in a lot of ways. There's in excellent explanation in the Administration manual. Rudy > > > Has anyone used Informix Data Replication extensively? I work for a > > > corporation who is considering using this Informix product and would > like to > > > find out specifically, how agent uses the logs to replicate and also > and > > > overhead or speed issues related to it's use. > > > > > > thanks > > Sent via Deja.com http://www.deja.com/ > Before you buy.