Urgent Online Optical Problem

This problem involves a process running on a 7.30.UC6-1 engine writing to a 5.07.UC1 system running OnLine Optical. Both machines run Solaris 2.6. We run three processes on the 7X machine. They have been running fine for months. As of the last 2 days they run successfully for 15-30 minutes and then die with either a 603 or 605 SQL and a 166 ISAM error. They can be restarted and run again for the same duration before erroring off again. It seems obvious that some resource is being used up, the process dies, resource freed, program restarted and so on. The 166 error mentions disk space and all is fine with the exception of my inability to monitor the STAGEBLOB space. Below are more details. -605 Cannot write blob. This statement writes a BYTE or TEXT value, and some unexpected error prevented the creation of the value. Roll back the current transaction. Check the accompanying ISAM error code for more information; possibly there has been a hardware error or data corruption of the blobspace or tblspace. One possible cause is that the blobspace for this column is full. Another is that although a new chunk has been assigned to the blobspace, pages cannot be allocated in it until the addition of the chunk has been logged and the log file closed. The OnLine administrator can force a log file to be closed using the tbmode -l or onmode -l command. If the error recurs, please note all circumstances and contact the Informix Technical Support Department. -603 Cannot close blob. This statement writes a BYTE or TEXT value, and some unexpected error prevented finishing the creation of the value. Roll back the current transaction. Check the accompanying ISAM error code for more information; possibly there has been a hardware error or data corruption of the blobspace or tblspace. If the error recurs, please note all circumstances and contact the Informix Technical Support Department. -166 ISAM err: BlobSpace is full. This operation attempts to insert or update the value of a BYTE or TEXT column, but there is not enough space in the blobspace in which that column is stored. The current transaction should be rolled back and the application terminated. Then contact the OnLine administrator and ask that a chunk of disk space be added to this blobspace. Note: When BYTE and TEXT values are deleted or replaced, the pages that they occupy in the blobspace do not become available for reuse until the logical log in which that transaction appears has been freed. A logical log has been freed if the log is backed up to tape and all transactions in the log are closed. The 5X machine has the optical storage. Space on the optical disk is fine. This instance has a STAGEBLOB dbspace set up on magnetic disk. tbstat -d never shows any of this space being used although OnLine writes to this before being written to optical. I manually ran the program in question with truss and noticed a 29, 22, 10 and 4 error in the output a portion of which follows: fcntl(3, F_GETFL, 0x00000000) = 2 fstat64(3, 0xEFFFE698) = 0 llseek(3, 0, SEEK_CUR) Err#29 ESPIPE ioctl(3, TCGETS, 0x0004D424) Err#22 EINVAL sigaction(SIGCLD, 0xEFFFE130, 0xEFFFE1B0) = 0 waitid(P_ALL, 0, 0xEFFFE170, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 sigaction(SIGCLD, 0xEFFFE130, 0xEFFFE1B0) = 0 Received signal #18, SIGCLD, in read() [caught] siginfo: SIGCLD CLD_EXITED pid=17816 status=0x0000 read(3, 0xEFFFE9B0, 1024) Err#4 EINTR setcontext(0xEFFFE0D0) sigaction(SIGCLD, 0xEFFFE570, 0xEFFFE5F0) = 0 waitid(P_ALL, 0, 0xEFFFE5B0, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) = 0 waitid(P_ALL, 0, 0xEFFFE5B0, WEXITED|WTRAPPED|WSTOPPED|WNOHANG) Err#10 ECHILD sigaction(SIGCLD, 0xEFFFE570, 0xEFFFE5F0) = 0 I would certainly appreciate any feedback on this problem. Thanks! Dick Brieck Chase Manhattan Mortgage Corporation