Re: System Freezes

I understand that your system freezing will hide any useful stuff from top or other tools like that. But if you take some snapshots (vmstat?) it may reveal something. Also... Is this a virtualized host? Are you sure no other host is "stealing" your CPU cores? (This would appear in the logs...) Wild guess... CGIs are nasty... Have you checked your netstat output? Do you have enough tcp ports configured? CGIs typically connec/disconnect, and if you have large TIME_WAIT parameters you can end up exhausting your available ports (it would block new connections, but should not affect existing ones). Are you using raw devices or file system? If it's fs, which one? As you may have noted, I'm shooting into the air... On Tue, Jan 11, 2011 at 10:26 PM, Rubinstein, James <JRUBIN@midwestern.edu>wrote: > Nothing of interest in our system log. The problems started Friday when I > turned on my new perl CGI’s (which do database operations), were no existent > over the weekend, when the scripts were not being used, and then started > again yesterday, when our university opened for business. I don’t notice > any freezes during weekends of evenings (when no one is working except for > me) and they start again when the web traffic and database activity starts > up again. > > > > *From:* informix-list-bounces@iiug.org [mailto: > informix-list-bounces@iiug.org] *On Behalf Of *Everett Mills > *Sent:* Tuesday, January 11, 2011 3:24 PM > > *Cc:* informix-list@iiug.org > *Subject:* RE: System Freezes > > > > Those symptoms sound like a hardware issue to me. Have you looked for > error messages in /var/adm/syslog/syslog.log? > > > > --EEM > > > > *From:* informix-list-bounces@iiug.org [mailto: > informix-list-bounces@iiug.org] *On Behalf Of *Fernando Nunes > *Sent:* Tuesday, January 11, 2011 4:07 PM > *To:* Rubinstein, James > *Cc:* informix-list@iiug.org > *Subject:* Re: System Freezes > > > > > > On Tue, Jan 11, 2011 at 9:29 PM, Rubinstein, James <JRUBIN@midwestern.edu> > wrote: > > I’m running IDS 11.50.FC6 on HPUX 11.31. We recently rolled out an > internally developed web-based system (Apache perl/mod_perl) and immediately > started noticing that our HPUX system becomes completely unresponsive, for > 20-30 seconds at a time, many times throughout the day. My first thought > was network problems, but we have pretty much ruled this out since I can > connect to a twin HPUX server which seems fine during the outages. During > these system freezes, any connections to the database fail and the system is > completely unresponsive to the point that I cannot even type any commands at > the shell. I have seen this behavior in the past when the oninit processes > use a lot of CPU resources, but it is usually pretty easy to track these > down to some bad SQL/report writing. In this case, I’m trying to figure out > what may be causing the system freezes. I have top and the dbtop utility > from IIUG, but those don’t refresh during or freezes. I am also unable to > type any onstat commands until the system comes back. By that time, > everything looks pretty normal with low load averages and our oninit > processes at normal levels. I’m looking at the various system reports in > OAT, but don’t see anything that jumps out as the culprit. I’d appreciate > any troubleshooting ideas. > > > > It may look as I'm defending Informix, but I find it very hard to believe > that the symptoms you describe can be caused by any kind of bad SQL. > I'd start looking at memory usage etc. Check the OS ratios for filesystem > cache vs program memory. Check you memory usage. Try to keep a "top" or > similar tool open and see if you notice something. Take frequent snapshots > of paging status (so that you can compare before/after counters. > Also check your system logs. > Last time I saw something similar (not on HP) it was the filesystem/program > memory ratios. If froze the machine whenever a filesystem intensive > operation was run. > > By no means I'm insinuating it's everything ok with Informix, but whatever > happens with it should not cause that effect. > > Regards. > > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently... > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently...