thread soctcplst in ready state

Hi, Environment: Informix 11.70.FC8W1 HP-UX 11.31 Today a large number of errors -25582 and -27001 have been reported on one of our instances. In the online log: listener-thread: err = -25582: oserr = 0: errstr = from <host.domain> to server <informixserver> : Network connection is broken. In the clients: -27001: Read error occurred during connection attempt I've noticed that sometimes one of the soctcplst threads stays in ready state for several seconds (30 - 60 seconds aprox). Example: onstat -g ath | grep lst 18 c00000170f294b10 0 2 ready 1cpu* soctcplst 21 c00000170f3a7c60 0 2 sleeping forever 8cpu* soctcplst During this period the connection attempts are frozen. With dbaccess, the connection freezes until it finally comes to connect (I don't know what is the timeout of the dbaccess). Other applications show error 27001. The host where the instance runs does not have high CPU load issues. In addition, other instances are running on this host and don't have this behaivour. I have dynamically added one VP soc and one VP cpu, but the problem continues. What causes the thread soctcplst to remain in ready state ? it's possible that it's due to a heavy sql running in the cpu number 1 ? Is there some way to separate the soctcplst threads from the sqlexec threads so that they don't run on the same VP cpu ? Please provide any clues according your experience. Thanks Best regards. Roger

Maybe a dns problem? dns reverse lookup may be slow / timing out, so accepting new connections must wait for the dns response. Check Fernando blog on this issue: http://informix-technology.blogspot.com/2012/01/dns-impact-on-informix-impacto-d o-dns.html Best regards, Luis Marques

Original post: Hi, Environment: Informix 11.70.FC8W1 HP-UX 11.31 Today a large number of errors -25582 and -27001 have been reported on one of our instances. In the online log: listener-thread: err = -25582: oserr = 0: errstr = from <host.domain> to server <informixserver> : Network connection is broken. In the clients: -27001: Read error occurred during connection attempt I've noticed that sometimes one of the soctcplst threads stays in ready state for several seconds (30 - 60 seconds aprox). Example: onstat -g ath | grep lst 18 c00000170f294b10 0 2 ready 1cpu* soctcplst 21 c00000170f3a7c60 0 2 sleeping forever 8cpu* soctcplst During this period the connection attempts are frozen. With dbaccess, the connection freezes until it finally comes to connect (I don't know what is the timeout of the dbaccess). Other applications show error 27001. The host where the instance runs does not have high CPU load issues. In addition, other instances are running on this host and don't have this behaivour. I have dynamically added one VP soc and one VP cpu, but the problem continues. What causes the thread soctcplst to remain in ready state ? it's possible that it's due to a heavy sql running in the cpu number 1 ? Is there some way to separate the soctcplst threads from the sqlexec threads so that they don't run on the same VP cpu ? Please provide any clues according your experience. Thanks Best regards. Roger Response A soctcplst thread is a high priority thread, the only reason for it to stay in a ready state for long periods of time (assuming you just aren't catching it when it hasn't had a chance to run, and this is easy to verify with onstat -g cpu output) is if there is another thread that is in a "running" state on that same cpu vp and that thread is not yielding properly (so it's holding on to the cpu vp too long). Again that can also be seen in the onstat -g cpu output if you look at the "Last Run" column and if the time is not being updated while the thread is running, that means it's running without yielding. Unfortunately, no there is no way to separate the listener threads from sqlexec threads, and in general it shouldn't be required because sqlexec threads should be coded to yield frequently so that higher priority threads, like the listener thread, can run. When that isn't the case, it's generally considered a defect, and there are known defects along those lines. What you would want to do at that point would be identify the thread that is running on that cpu vp and see what sql that session is doing, and also a good idea would be to use some system tool like procstack or pstack to get a couple stack traces from the cpu vp, which can then also be beneficial for support to try and then map this to any known defects. Jacques Renaut HCL Informix Advanced Support

Thanks. dns is working properly. Regards.

Hi Jacques, In effect, we did the research and found that the day before an application had been put into production that had a defect. The defect was that it did an SQL query with insufficient filters of a large table, something like select * from <bigtable> order by .. This application run from an IIS pool so that several queries were generated at the same time. The application was fixed and the "ready" state is no longer observed. Now almost all the time the state of soctcplst threads is "sleeping forever". Thank you very much. Regards Roger