How can I tell what's eating my global memory pool?
Guys-
We have a pair of (supposed to be) identical IDS servers running
in two of our plants. They have similar software, identical hardware,
etc. They are both running IDS 7.31.FD4 on HP-UX 11.0. The less busy
one, however has a global memory pool that grows until it chews up all
of the memory allocated to informix.
Here's what the memory usage on the sick server looks like:
Informix Dynamic Server Version 7.31.FD4 -- On-Line (Prim) -- Up
13 days 17:
57:44 -- 317504 Kbytes
Segment Summary:
id key addr size ovhd class
blkused bl
kfree
4100 1381451777 c00000000023c000 156409856 36048 R*
19084 9
1543 1381451780 c000000009766000 167772160 3200 V*
7526 12
954
7178 1381451783 c000000013766000 942080 656 M
108 7
Total: - - 325124096 - -
26718 12
970
Here's the well one:
Informix Dynamic Server Version 7.31.FD4 -- On-Line (Prim) -- Up
77 days 16:
18:11 -- 317504 Kbytes
Segment Summary:
id key addr size ovhd class
blkused bl
kfree
6660 1381451777 c00000000023c000 156409856 36048 R*
19084 9
7175 1381451780 c000000009766000 167772160 3200 V*
4059 16
421
1546 1381451783 c000000013766000 942080 656 M
108 7
Total: - - 325124096 - -
23251 16
437
(* segment locked in memory)
Note that the less busy server has in use 7526 blocks of virtual
segment memory vs. 4059 in the good server (snapshot at peak usage).
Tomorrow the bad one will probably jump to about 8500, while the other
continues at 3500-4000. I recently discovered that the global memory
pool is where this is going. Here's an example (onstat -g mem | grep
global over a few days):
Sick Server:
Pool Summary:
name class addr totalsize freesize #allocfrag
#freefrag
8/8/03
global V c00000000976e028 35987456 4511120 28294
13396
8/11/03
global V c00000000976e028 44130304 5764160 35748
17361
8/12/03
global V c00000000976e028 46628864 6142168 38397
18653
Good Server:
Pool Summary:
name class addr totalsize freesize #allocfrag
#freefrag
8/8/03
global V c00000000976e028 11026432 689096 3492 558
8/11/03
global V c00000000976e028 11034624 746872 3304 548
8/12/03
global V c00000000976e028 11132928 688456 3711 498
Note how the sick server's global pool grows daily. The question is:
How can I track down what's eating that up. The Docs only references
to it are vague at best.
--EEM