HDR- Distance and Latency
Posted in 2017
Topics: High Availability & Replication, Performance & Tuning, Platform-Specific Issues
We have our production and DR (disaster) data center 40 miles apart in the Chicago area. Informix production data is replicating to DR using HDR. Now our DR servers are moving to Austin, TX almost 1200 miles away. Informix 12.10.FC6 on HP-UX 11.31 ia64 Questions: 1. Is there significant ( > 100%) latency with HDR? Is there any (performance) impact of this latency on the Primary? What should I expect? The Secondary is read-only. 2. Are there tunable config parameters to minimize latency with our configuration? 3. With DR moving to Austin, is HDR the right technology. Should I be looking at other mechanisms to replicate informix- perhaps 2 updateable servers staying in sync Thank you.
HDR over a WAN is usually not workable. It depends on how fast the connection to the DR site is and what the network latency is. Yes, delays in applying log buffers sent to the HDR secondary can affect the performance of the primary as far as clients committing transactions are concerned. Even in near-sync mode the primary must wait to acknowledge a COMMIT request until the secondary acknowledges that it safely received and saved the logical log buffer to disk before it can return to the client. At a client of mine recently we had just this problem. Latency to the DR site was around 40ms and client apps were experiencing delays and slow performance. Four minute batch jobs were taking over an hour to complete. We did what I am going to recommend that you do. Change the remote secondary for DR to an RSS secondary and optionally plan to install a local HDR secondary in your primary data center as the first line of failover. With the RSS secondary running those batch jobs are back to completing in ~4mins. Art Art S. Kagel, President and Principal Consultant ASK Database Management www.askdbmgt.com Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Jul 19, 2017 at 8:40 AM, MURALI PAZHAYANNUR < pmurali@ftportfolios.com> wrote: > We have our production and DR (disaster) data center 40 miles apart in the > Chicago area. Informix production data is replicating to DR using HDR. Now > our > DR servers are moving to Austin, TX almost 1200 miles away. > > Informix 12.10.FC6 on HP-UX 11.31 ia64 > > Questions: > 1. Is there significant ( > 100%) latency with HDR? Is there any > (performance) > impact of this latency on the Primary? What should I expect? The Secondary > is > read-only. > 2. Are there tunable config parameters to minimize latency with our > configuration? > 3. With DR moving to Austin, is HDR the right technology. Should I be > looking > at other mechanisms to replicate informix- perhaps 2 updateable servers > staying in sync > Thank you. > > > ************************************************************ > ******************* > Forum Note: Use "Reply" to post a response in the discussion forum. > >
here's my story ... IDS 11.50.FC8. Dallas TX to Wilmington DE (> 1000 miles). At the busiest times (over 40 commits per second) the primary would block waiting on the log latch. R&D has corrected that issue in later versions I believe , not totally sure on since we moved to using ER instead on 11.70.FC7 . So yes.. latency does come into play and you have to also consider your internet business provider and the level of service they provide. Network congestion will torpedo your HDR primary, no doubt about it. RSS was not an option due to the SLA agreements in place but if you can live with the async that RSS comes with then consider that as an option. After switching to ER, we do get occasional ER spooling due to network latency/spooling but at least the "primary" does not block ,most important. You can tune HDR on the secondary by minimizing the checkpoint duration's (they block the primary to ensure data consistency) , see LRU MAX & MIN tuning and setting DRINTERVAL accordingly. Finally there is this with HDR: If your network is not very reliable (in other works just like any other network) occasionally the secondary will detect that the primary is not online and not re-establish connectivity automatically due to the SMX thread exiting which in that case a restart of the secondary is needed to get them talking again. But there will be times where that will not work and you have to restart the engine on the primary to re-establish connectivity. I opened several PMRs over this and nothing ever became of it, its a definite pain point. IMO HDR is a viable solution if the servers are relatively close together but if they are not (such as 1000+ miles apart) and if the primary is highly OLTP then start kicking the tires on ER or RSS. Mark
A couple of additional points. HDR communication is half duplexed over a single network connection while RSS ( SMX) is using full duplexed mode. Additionslly SMX can run in parallel over multiple network connections resulting is a significant boost to effective bandwidth. Sent from Yahoo Mail on Android On Wed, Jul 19, 2017 at 9:21 AM, Art Kagel<art.kagel@gmail.com> wrote: HDR over a WAN is usually not workable. It depends on how fast the connection to the DR site is and what the network latency is. Yes, delays in applying log buffers sent to the HDR secondary can affect the performance of the primary as far as clients committing transactions are concerned. Even in near-sync mode the primary must wait to acknowledge a COMMIT request until the secondary acknowledges that it safely received and saved the logical log buffer to disk before it can return to the client. At a client of mine recently we had just this problem. Latency to the DR site was around 40ms and client apps were experiencing delays and slow performance. Four minute batch jobs were taking over an hour to complete. We did what I am going to recommend that you do. Change the remote secondary for DR to an RSS secondary and optionally plan to install a local HDR secondary in your primary data center as the first line of failover. With the RSS secondary running those batch jobs are back to completing in ~4mins. Art Art S. Kagel, President and Principal Consultant ASK Database Management www.askdbmgt.com Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, Jul 19, 2017 at 8:40 AM, MURALI PAZHAYANNUR < pmurali@ftportfolios.com> wrote: > We have our production and DR (disaster) data center 40 miles apart in the > Chicago area. Informix production data is replicating to DR using HDR. Now > our > DR servers are moving to Austin, TX almost 1200 miles away. > > Informix 12.10.FC6 on HP-UX 11.31 ia64 > > Questions: > 1. Is there significant ( > 100%) latency with HDR? Is there any > (performance) > impact of this latency on the Primary? What should I expect? The Secondary > is > read-only. > 2. Are there tunable config parameters to minimize latency with our > configuration? > 3. With DR moving to Austin, is HDR the right technology. Should I be > looking > at other mechanisms to replicate informix- perhaps 2 updateable servers > staying in sync > Thank you. > > > ************************************************************ > ******************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.