Memory errors in online.log when the esql/c program does exit

Hi, I see messages as the following in online.log: 21:02:56 Found during mt_shm_free 1, error seen 1 times since last print to the log 21:02:56 Pool '328' (0xc000000087ed8040) 21:02:56 Bad block header 0xc000000090c2ef70 21:02:56 Assert Warning: Memory block header corruption detected in mt_shm_free 1 21:02:56 IBM Informix Dynamic Server Version 12.10.FC15W1 21:02:56 Who: Session(328, informix@vpar3si, 10997, c0000000855fed40) Thread(493, sqlexec, c0000000855c74d0, 8) File: mtshpool.c Line: 2806 21:02:56 Results: Pool repaired 21:02:56 Action: Please notify IBM Informix Technical Support. 21:02:56 stack trace for pid 10289 written to /opt/informix12.10FC15W1/tmp/af.5d5fa50 21:02:56 See Also: /opt/informix12.10FC15W1/tmp/af.5d5fa50 21:02:56 Memory block header corruption detected in mt_shm_free 1 21:04:58 Found during mt_shm_free 1, error seen 1 times since last print to the log 21:04:58 Pool '379' (0xc000000088c27040) 21:04:58 Bad block header 0xc000000090a58000 21:04:58 Assert Warning: Memory block header corruption detected in mt_shm_free 1 21:04:58 IBM Informix Dynamic Server Version 12.10.FC15W1 21:04:58 Who: Session(379, informix@vpar3si, 11059, c000000085603200) Thread(546, sqlexec, c0000000855d1490, 8) File: mtshpool.c Line: 2806 21:04:58 Results: Pool repaired 21:04:58 Action: Please notify IBM Informix Technical Support. 21:04:58 stack trace for pid 10289 written to /opt/informix12.10FC15W1/tmp/af.60afaca 21:04:58 See Also: /opt/informix12.10FC15W1/tmp/af.60afaca 21:04:58 Memory block header corruption detected in mt_shm_free 1 21:04:58 Assert Failed: Memory freed at bad boundary. 1 blocks at c000000090a58400 21:04:58 IBM Informix Dynamic Server Version 12.10.FC15W1 21:04:58 Who: Thread(546, sqlexec, c0000000855d1490, 8) File: mtshblk.c Line: 527 21:04:58 Results: Memory lost 21:04:58 Action: Please notify IBM Informix Technical Support. 21:04:58 stack trace for pid 10289 written to /opt/informix12.10FC15W1/tmp/af.60afaca 21:04:58 See Also: /opt/informix12.10FC15W1/tmp/af.60afaca 21:04:58 Memory freed at bad boundary. 1 blocks at c000000090a58400I have realized that this messages appear in online.log just when a esql/c program exits (that means the C code say "exit(0);"). The program doesnt show any error. Apparently it is not related to a particular esql/c program. I have not noticed if this was happening before, but recently this is having an impact because after some days the instance hang and the error: memory allocation failed is raised. Please provide us any information about the possibles causes of this bug. I had searching in the web and I found similar errors, but these errors are supposedly corrected in the version 12.10FC15. My environment: HP-UX 11.31 64 bits Informix Server 12.10FC15W1 CSDK 4.10.FC15 Regards, Roger ------------------------------ Roger Vilca ------------------------------

Hi Roger, when client programs disconnect/terminate, the associated back end session would perform cleanup and then terminate itself too. The fact that these assertions occur at client termination would indicate that some memory corruption is detected during session cleanup, within the respective sessions' memory pools (pool '328' when session 328 terminates, pool '379' for session 379). Apparently the corruption hadn't caused any problem while the session still was operational, and at this final stage of the session none of these errors are (can be) reported back to the client (hence no client side errors). As long as things stay this way, esp. corruptions remaining 'private' to sessions and sessions only corrupting their own session memory, the system probably will be able to handle these problems and no cross-session or server wide impact might occur, so the assertions will not be fatal and the server can continue normally. Any further analysis belongs into a support case. With some luck af files will be sufficient to determine a known defect's signature, otherwise a shared memory dump will be required or even some traces will have to be enabled (or added). A core question will be: what's changed, on server and/or application side, that might be contributing to this; any new functionality being used, any new SQL constructs, etc... HTH, Andreas ------------------------------ Andreas Legner ------------------------------

Roger: I am guessing that these ESQL/C applications are connecting to the server over shared memory, otherwise they are unlikely to be able to corrupt server memory. If so, you may be able to identify whether this is an application bug or a server bug by running the apps using a network connection. If the corruption is being caused by some server bug then changing to a networked connection type will not eliminate the issue in the server and you will still see those messages when the applications exit. If it is an application problem then the server messages will go away but the applications may begin to fail with their own memory issues instead. If that happens you may then be able to run them in a debugger and track the issue down. Art ------------------------------ Art S. Kagel, President and Principal Consultant ASK Database Management Corp. www.askdbmgt.com ------------------------------

Does esqlc code call any C UDRs ? I have seen something similar when the C UDR cleans up and the session closes

Hi Andreas, thanks for your response. The server has no changed, neither the application, as far as I have been able to review. Currently, I have the following values related to shared memory dump, what values do you recommend ? DUMPDIR $INFORMIXDIR/tmp DUMPSHMEM 0 DUMPGCORE 0 DUMPCORE 0 DUMPCNT 10 Thanks,Roger ------------------------------ Roger Vilca ------------------------------

Hello Art, In this case the client and the server are running over the same host, but the connection is using networked connection. Regards, Roger ------------------------------ Roger Vilca ------------------------------

Hello Paul, The esql is not using UDRs. The only notorious that I have noticed is that this program use several prepare and execute sentences for update and delete sentences with USING clause, but I think this should be usual in esql programs. Regards, Roger ------------------------------ Roger Vilca ------------------------------

DUMPSHMEM 2This will produce a shmem dump omitting the bufferpool(s), so significantly smaller and sufficient typically for this type of memory corruption. (Never turn on DUMPCORE or DUMPGCORE if not very explicitly told to do by tech support.) DUMPCNT 1 should be sufficient.And, of course, DUMPDIR should offer enough room (and not sit on your root file system, esp. if not much space left). You can modify all these using 'onmode -w{m|f}', so no need for a restart. BR, Andreas ------------------------------ Andreas Legner ------------------------------

Not all conscripts are defined as juridical to computer science whence they do not define proper html to construe to CRM as deputy. http://cryphosnegative.blogspot.com/ ------------------------------ KIN BUENO ------------------------------