Lock Table Overflow

IDS 7.31.UD2 AIX 5.2.0.0 (ml 2) Twice today we have encountered Lock Table Overflows which have caused many problems for us. We use the log_full.sh script as a basis for trapping certain information when this occurs (using ALARM type 21 - Resource Overflow), and when we have the diagnostic information from the first time that the log_full script is run we tell the log_full script to exit. However it appears that Informix is spawning many more log_full scripts than the Operating System can cope with. We have MAX Process set to 256 (per user, excluding root) and before each script is getting to the exit point we have exceeded the 256 process limit and the server starts to struggle. We receive message "0403-030 The fork function failed. Too many processes already exist" when trying to su to informix. Do you know if there are any values we can set within Informix to combat this, (obviously we need to address the resource overflow in the application).

Hi, I would change the log_full script to run only once. This could be established by trying to read the process-id In a file (like e.g. apache does it in apachectl). Store this in /var/run/log_full.pid or whereever you like. If this process is still existing, (ps or a cat in /proc, or something similar), just exit. Else store your own process-id ($$) in this file and proceed. At end of the script, delete the file. You might lose some information, but you will likely prevent the max process limit. Also, set NUMLOCKS to a higher value ... Marcus -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of EILEEN LITTLER Sent: Tuesday, May 16, 2006 5:47 PM To: ids@iiug.org Subject: Lock Table Overflow [6733] IDS 7.31.UD2 AIX 5.2.0.0 (ml 2) Twice today we have encountered Lock Table Overflows which have caused many problems for us. We use the log_full.sh script as a basis for trapping certain information when this occurs (using ALARM type 21 - Resource Overflow), and when we have the diagnostic information from the first time that the log_full script is run we tell the log_full script to exit. However it appears that Informix is spawning many more log_full scripts than the Operating System can cope with. We have MAX Process set to 256 (per user, excluding root) and before each script is getting to the exit point we have exceeded the 256 process limit and the server starts to struggle. We receive message "0403-030 The fork function failed. Too many processes already exist" when trying to su to informix. Do you know if there are any values we can set within Informix to combat this, (obviously we need to address the resource overflow in the application). **************************************************************************** *** Forum Note: Use "Reply" to post a response in the discussion forum.

No, it fires the alarm program each time a condition occurs. Have the alarm progrsm touch a file that a deamon can then notice and do the required actions. Then the alarmprogram finishes quickly (could even convert it into a C program to be faster). The daemon can then loop doing - sleep 10 seconds - if file exists do action remove file. You can even have a directory that can contain different files for different actions David. ----- Original Message ----- From: "EILEEN LITTLER" <eileen.littler@pilkington.com> To: <ids@iiug.org> Sent: Tuesday, May 16, 2006 4:47 PM Subject: Lock Table Overflow [6733] > > IDS 7.31.UD2 AIX 5.2.0.0 (ml 2) > Twice today we have encountered Lock Table Overflows which have caused > many > problems for us. > > We use the log_full.sh script as a basis for trapping certain information > when > this occurs (using ALARM type 21 - Resource Overflow), and when we have > the > diagnostic information from the first time that the log_full script is run > we > tell the log_full script to exit. However it appears that Informix is > spawning > many more log_full scripts than the Operating System can cope with. We > have > MAX Process set to 256 (per user, excluding root) and before each script > is > getting to the exit point we have exceeded the 256 process limit and the > server starts to struggle. We receive message "0403-030 The fork function > failed. Too many processes already exist" when trying to su to informix. > > Do you know if there are any values we can set within Informix to combat > this, > (obviously we need to address the resource overflow in the application). > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --- > [This E-mail has been scanned for viruses but it is your responsibility > to maintain up to date anti virus software on the device that you are > currently using to read this email. ] > > > > > -- > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.392 / Virus Database: 268.5.6/340 - Release Date: 15/05/2006 > > --- [This E-mail has been scanned for viruses but it is your responsibility to maintain up to date anti virus software on the device that you are currently using to read this email. ]

Get my package utils3_ak from the IIUG Software Repository and compile and install the eventalarm.c in ALARMPROGRAM instead of log_full.sh. It will handle multiply occurring events like the lock table overflow event by ignoring all by the first to arrive in a given configurable period of time. The config file, eventalarm.cfg, is used to change the time period over which multiple events are ignored. Eventalarm also has other functionality. As it is currently coded it will archive each logical log as it fills using ontape -a and email a list of addresses when critical events occur. The mulitply occurring events feature causes the program to exit very quickly, unlike log_full.sh, if a multiple event is detected and will send only one email during each of the ignore periods if the events continue for longer than the configure time period (30 minutes in the sample config file). Eventalarm.c also detects when another copy of itself is already running and also exits quickly to allow the prior copy to handle the event. Art S. Kagel ----- Original Message ----- From: Eileen Littler <ids@iiug.org> At: 5/16 11:50:52 IDS 7.31.UD2 AIX 5.2.0.0 (ml 2) Twice today we have encountered Lock Table Overflows which have caused many problems for us. We use the log_full.sh script as a basis for trapping certain information when this occurs (using ALARM type 21 - Resource Overflow), and when we have the diagnostic information from the first time that the log_full script is run we tell the log_full script to exit. However it appears that Informix is spawning many more log_full scripts than the Operating System can cope with. We have MAX Process set to 256 (per user, excluding root) and before each script is getting to the exit point we have exceeded the 256 process limit and the server starts to struggle. We receive message "0403-030 The fork function failed. Too many processes already exist" when trying to su to informix. Do you know if there are any values we can set within Informix to combat this, (obviously we need to address the resource overflow in the application). ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

Art, Do you have any plans to code this for onbar when the logs fill? "ART KAGEL, ...." <kagel@bloomberg.net> Sent by: ids-bounces@iiug.org 05/16/2006 12:23 PM Please respond to ids@iiug.org To ids@iiug.org cc Subject Re: Lock Table Overflow [6738] Get my package utils3_ak from the IIUG Software Repository and compile and install the eventalarm.c in ALARMPROGRAM instead of log_full.sh. It will handle multiply occurring events like the lock table overflow event by ignoring all by the first to arrive in a given configurable period of time. The config file, eventalarm.cfg, is used to change the time period over which multiple events are ignored. Eventalarm also has other functionality. As it is currently coded it will archive each logical log as it fills using ontape -a and email a list of addresses when critical events occur. The mulitply occurring events feature causes the program to exit very quickly, unlike log_full.sh, if a multiple event is detected and will send only one email during each of the ignore periods if the events continue for longer than the configure time period (30 minutes in the sample config file). Eventalarm.c also detects when another copy of itself is already running and also exits quickly to allow the prior copy to handle the event. Art S. Kagel ----- Original Message ----- From: Eileen Littler <ids@iiug.org> At: 5/16 11:50:52 IDS 7.31.UD2 AIX 5.2.0.0 (ml 2) Twice today we have encountered Lock Table Overflows which have caused many problems for us. We use the log_full.sh script as a basis for trapping certain information when this occurs (using ALARM type 21 - Resource Overflow), and when we have the diagnostic information from the first time that the log_full script is run we tell the log_full script to exit. However it appears that Informix is spawning many more log_full scripts than the Operating System can cope with. We have MAX Process set to 256 (per user, excluding root) and before each script is getting to the exit point we have exceeded the 256 process limit and the server starts to struggle. We receive message "0403-030 The fork function failed. Too many processes already exist" when trying to su to informix. Do you know if there are any values we can set within Informix to combat this, (obviously we need to address the resource overflow in the application). ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum. ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.

I don't use onbar, but I imagine that the changes to eventalarm.c to support onbar instead are not difficult. It would be changes to the dialog in 'case 23:' in the switch(eveseve) in main() and the sprintf & execlp at lines 436 & 437. Art S. Kagel ----- Original Message ----- From: Peter_logan.... <ids@iiug.org> At: 5/16 13:20:14 Art, Do you have any plans to code this for onbar when the logs fill? "ART KAGEL, ...." <kagel@bloomberg.net> Sent by: ids-bounces@iiug.org 05/16/2006 12:23 PM Please respond to ids@iiug.org To ids@iiug.org cc Subject Re: Lock Table Overflow [6738] Get my package utils3_ak from the IIUG Software Repository and compile and install the eventalarm.c in ALARMPROGRAM instead of log_full.sh. It will handle multiply occurring events like the lock table overflow event by ignoring all by the first to arrive in a given configurable period of time. The config file, eventalarm.cfg, is used to change the time period over which multiple events are ignored. Eventalarm also has other functionality. As it is currently coded it will archive each logical log as it fills using ontape -a and email a list of addresses when critical events occur. The mulitply occurring events feature causes the program to exit very quickly, unlike log_full.sh, if a multiple event is detected and will send only one email during each of the ignore periods if the events continue for longer than the configure time period (30 minutes in the sample config file). Eventalarm.c also detects when another copy of itself is already running and also exits quickly to allow the prior copy to handle the event. Art S. Kagel ----- Original Message ----- From: Eileen Littler <ids@iiug.org> At: 5/16 11:50:52 IDS 7.31.UD2 AIX 5.2.0.0 (ml 2) Twice today we have encountered Lock Table Overflows which have caused many problems for us. We use the log_full.sh script as a basis for trapping certain information when this occurs (using ALARM type 21 - Resource Overflow), and when we have the diagnostic information from the first time that the log_full script is run we tell the log_full script to exit. However it appears that Informix is spawning many more log_full scripts than the Operating System can cope with. We have MAX Process set to 256 (per user, excluding root) and before each script is getting to the exit point we have exceeded the 256 process limit and the server starts to struggle. We receive message "0403-030 The fork function failed. Too many processes already exist" when trying to su to informix. Do you know if there are any values we can set within Informix to combat this, (obviously we need to address the resource overflow in the application). ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum. ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum. ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.