Trouble in HDR land
Posted in 2014
Topics: High Availability & Replication, Performance & Tuning
Ok, I have an unusually situation:
Informix 11.7Fc7
Environment: ZVM Mainframe, zVm with Suse 11.2 , 8GB memory allocated, 2 CPU's
allocated. File system EXT3.
Running MulitInstance on cooked file systems.
FICON back to 8300 SAN unit with 4 Ficon adaptors
10 instance in test LPAR
same instances replicated on HDR
No connections, just debugging HDR.
Primary runs fine, shows less than 0.10 total system load with all 10 instances
running. Mostly stays in the .01 , .02 load average.
HDR secondary connects fine. Starts off at .10 load average after about 10
minutes HDR load average starts hitting 1.0 - 2.0 (yup 100 - 200 percent load
average).
SAR shows CPU usage on replica is staying 60 - 80% idle, yet average load is
going crazy.
Any tuning suggestions on the HDR? Any where I should look (onstat commands?)
DRINTERVAL set to -1, drives it further north. Setting DRINTERVAL to 0 or 1
seems to help, any higher and no difference.
We have switched to jumbo frames , 8992, which helped a little.
Any help would be appreciated ...
Yeah, yeah, I know ART does not like MI or cooked file systems, but it is not
an option here.
Thanks
--
Ben Duncan - Business Network Solutions, Inc. 336 Elton Road Jackson MS, 39212
"Never attribute to malice, that which can be adequately explained by
stupidity"
- Hanlon's Razor
Hi Ben.
Don't using journaled file systems: move to EXT2 or EXT4 with journaling off.
This is still the best presentation on HDR by Ron Privett at IIUG 2010:
https://dl.dropboxusercontent.com/u/97610754/HDR_Best_Practices_and_Performance_
Tuning.pdf
For example, you could increase the TCP buffer size: 32K always works (default
is 4K), i.e. append "b=32768" to the entries in "sqlhosts" used for HDR.
Do you have particular database objects being thrashed on the secondary? Look
in sysmaster:sysptprof and compare with the primary. There are many "onstat"
commands to investigate further if so to see which threads are involved.
I'm sure others will give lots more advice!
Regards,
Doug Lawry