SCHAPI post_alarm_messages

I migrated from IDS 11.50 on HPUX to IDS 11.70.FC4 on SLES 11 SP 2 a few weeks ago. Since that time, I¹m seeing periodic messages like this in the online logs: 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not delete a row. 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM error: deadlock detected Is this related to something running in the scheduler? I do see this task from sysadmin:ph_task: tk_id 1 tk_name m on_command_history tk_description Monitor how much data is kept in the command history table tk_type TASK tk_sequence 72 tk_result_table tk_create tk_dbs sysadmin tk_execute delete from command_history where cmd_exec_time < ( select current - value::INTERVAL DAY to SECOND from ph_threshold where name = 'COMMAND HISTORY RETENTION' ) tk_delete 0 01:00:00 tk_start_time 02:00:00 tk_stop_time tk_frequency 1 00:00:00 tk_next_execution 2013-04-26 09:54:51 tk_total_executio+ 72 tk_total_time 0.10489027422 What is the best way to troubleshoot where this error is coming from? Here is the full history of this error over the past couple of weeks: 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -240 Could not delete a row. 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -143 ISAM error: deadlock detected 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -240 Could not delete a row. 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -143 ISAM error: deadlock detected 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -240 Could not delete a row. 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -143 ISAM error: deadlock detected 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -240 Could not delete a row. 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -143 ISAM error: deadlock detected 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -240 Could not delete a row. 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -143 ISAM error: deadlock detected 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -240 Could not delete a row. 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -143 ISAM error: deadlock detected 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -240 Could not delete a row. 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -143 ISAM error: deadlock detected 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not delete a row. 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM error: deadlock detected

Hello. Have you tried to rebuild the sysadmin after the upgrade? There were some issues where the inplace upgrade did not upgrade sysadmin database, and you have to do it manually as a workaround. (Backup any of your additional tasks, if you have some, before doing it, it will be build with all defaults, only). Hope it helps. Regards. Alexandre Marini IBM Informix Certified Professional v10 / v11.50 / v11.70 IBM Information Management Informix Technical Professional IBM Infosphere DataStage Technical Professional Informix Senior DBA - Orizon Brasil BRIUG website administrator Informix independent consultant > To: ids@iiug.org > From: JRUBIN@midwestern.edu > Subject: SCHAPI post_alarm_messages [30137] > Date: Thu, 25 Apr 2013 13:12:22 -0400 > > I migrated from IDS 11.50 on HPUX to IDS 11.70.FC4 on SLES 11 SP 2 a few > weeks ago. Since that time, I¹m seeing periodic messages like this in the > online logs: > > 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not delete > a row. > 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM error: > deadlock detected > > Is this related to something running in the scheduler? I do see this task > from sysadmin:ph_task: > > tk_id 1 > tk_name m on_command_history > tk_description Monitor how much data is kept in the command history table > tk_type TASK > tk_sequence 72 > tk_result_table > tk_create > tk_dbs sysadmin > tk_execute delete from command_history where cmd_exec_time < ( > > select current - value::INTERVAL DAY to SECOND > > from ph_threshold > > where name = 'COMMAND HISTORY RETENTION' ) > tk_delete 0 01:00:00 > tk_start_time 02:00:00 > tk_stop_time > tk_frequency 1 00:00:00 > tk_next_execution 2013-04-26 09:54:51 > tk_total_executio+ 72 > tk_total_time 0.10489027422 > > What is the best way to troubleshoot where this error is coming from? > > Here is the full history of this error over the past couple of weeks: > > 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -240 Could > not delete a row. > 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -143 ISAM > error: deadlock detected > > 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -240 Could > not delete a row. > 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -143 ISAM > error: deadlock detected > > 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -240 Could > not delete a row. > 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -143 ISAM > error: deadlock detected > > 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -240 Could > not delete a row. > 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -143 ISAM > error: deadlock detected > > 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -240 Could > not delete a row. > 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -143 ISAM > error: deadlock detected > > 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -240 Could > not delete a row. > 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -143 ISAM > error: deadlock detected > > 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -240 Could > not delete a row. > 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -143 ISAM > error: deadlock detected > > 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could > not delete a row. > 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM > error: deadlock detected > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. >

I haven't tried to rebuild sysadmin yet, but I=92ve looked into this a = bit more. What is happening is that the error occurs after a logical = log is filled and is backed up. In alarmprog.sh, we=92ve set the = following: BACKUPLOGS=3DY BACKUP_CMD=3D"ontape -a -d" We=92ll go days backup logical logs as they fill, but on occasion we=92ll = get an error like this: 16:40:12 Logical Log 890 Complete, timestamp: 0x8dc8c8e3. 16:40:12 Logical Log 890 - Backup Started 16:40:12 Logical Log 890 - Backup Completed 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -240 Could not = delete a row. 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -143 ISAM error: = deadlock detected The error is with the post_alarm_message task, which executes the = ph_dbs_alert command. I see the following routine in sysadmin: create function informix.ph_dbs_alert(informix.integer, = informix.integer, informix.pointer) returns integer external name '(ph_dbs_alert)' language C; What I haven't figured out is what the database is trying to delete. Do = you think a rebuild of sysadmin is still needed or is there a way to = better try to figure out what is happening as the logical logs fill and = alarmprog.sh is run? I should note that we haven=92t modified = alarmprog.sh, except for the two lines mentioned above. Since March 30, = when we migrated to our new server, we have backed up 890 logical logs. = This error has after 11 of the backups when = alarmprog.sh/post_alarm_message/ph_dbs_alert was executed. =20 > From: Alexandre Marini <alexandre@briug.org> > Subject: RE: SCHAPI post_alarm_messages [30138] > Date: April 25, 2013 10:38:08 AM MST > To: <ids@iiug.org> > Reply-To: <ids@iiug.org> >=20 >=20 > Hello.=20 > Have you tried to rebuild the sysadmin after the upgrade? There were = some=20 > issues where the inplace upgrade did not upgrade sysadmin database, = and you=20 > have to do it manually as a workaround.=20 >=20 > (Backup any of your additional tasks, if you have some, before doing = it, it=20 > will be build with all defaults, only).=20 >=20 > Hope it helps.=20 > Regards.=20 >=20 > Alexandre Marini=20 > IBM Informix Certified Professional v10 / v11.50 / v11.70=20 >=20 > IBM Information Management Informix Technical Professional=20 >=20 > IBM Infosphere DataStage Technical Professional=20 > Informix Senior DBA - Orizon Brasil=20 > BRIUG website administrator=20 > Informix independent consultant=20 >=20 >> To: ids@iiug.org=20 >> From: JRUBIN@midwestern.edu=20 >> Subject: SCHAPI post_alarm_messages [30137]=20 >> Date: Thu, 25 Apr 2013 13:12:22 -0400=20 >>=20 >> I migrated from IDS 11.50 on HPUX to IDS 11.70.FC4 on SLES 11 SP 2 a = few=20 >> weeks ago. Since that time, I=B9m seeing periodic messages like this = in the=20 >> online logs:=20 >>=20 >> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not = delete=20 >> a row.=20 >> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM error:=20= >> deadlock detected=20 >>=20 >> Is this related to something running in the scheduler? I do see this = task=20 >> from sysadmin:ph_task:=20 >>=20 >> tk_id 1=20 >> tk_name m on_command_history=20 >> tk_description Monitor how much data is kept in the command history = table=20 >> tk_type TASK=20 >> tk_sequence 72=20 >> tk_result_table=20 >> tk_create=20 >> tk_dbs sysadmin=20 >> tk_execute delete from command_history where cmd_exec_time < (=20 >>=20 >> select current - value::INTERVAL DAY to SECOND=20 >>=20 >> from ph_threshold=20 >>=20 >> where name =3D 'COMMAND HISTORY RETENTION' )=20 >> tk_delete 0 01:00:00=20 >> tk_start_time 02:00:00=20 >> tk_stop_time=20 >> tk_frequency 1 00:00:00=20 >> tk_next_execution 2013-04-26 09:54:51=20 >> tk_total_executio+ 72=20 >> tk_total_time 0.10489027422=20 >>=20 >> What is the best way to troubleshoot where this error is coming from?=20= >>=20 >> Here is the full history of this error over the past couple of weeks:=20= >>=20 >> 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >> 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >> 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >> 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >> 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >> 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >> 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >> 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 = Could=20 >> not delete a row.=20 >> 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 = ISAM=20 >> error: deadlock detected=20 >>=20 >>=20 >>=20 > = **************************************************************************= *****=20 >> Forum Note: Use "Reply" to post a response in the discussion forum.=20= >>=20 >=20 >=20 > = **************************************************************************= *****=20 > Forum Note: Use "Reply" to post a response in the discussion forum.=20= >=20 =20

Is it possible that at those times when the error has occurred two logs completed very close on each other's heels? I also don't know what would be being deleted, but it looks like you have a task scheduler sensor detecting ALARM events and it is something being done by that task scheduler procedure. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, May 8, 2013 at 6:36 PM, Jim Rubinstein <jrubin@midwestern.edu>wrote: > I haven't tried to rebuild sysadmin yet, but I=92ve looked into this a = > bit more. What is happening is that the error occurs after a logical = > log is filled and is backed up. In alarmprog.sh, we=92ve set the = > following: > > BACKUPLOGS=3DY > BACKUP_CMD=3D"ontape -a -d" > > We=92ll go days backup logical logs as they fill, but on occasion we=92ll = > get an error like this: > > 16:40:12 Logical Log 890 Complete, timestamp: 0x8dc8c8e3. > 16:40:12 Logical Log 890 - Backup Started > 16:40:12 Logical Log 890 - Backup Completed > 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -240 Could not = > delete a row. > > 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -143 ISAM error: = > deadlock detected > > The error is with the post_alarm_message task, which executes the = > ph_dbs_alert command. I see the following routine in sysadmin: > > create function informix.ph_dbs_alert(informix.integer, = > informix.integer,> > informix.pointer) > > returns integer > > external name '(ph_dbs_alert)' > > language C; > > What I haven't figured out is what the database is trying to delete. Do = > you think a rebuild of sysadmin is still needed or is there a way to = > better try to figure out what is happening as the logical logs fill and = > alarmprog.sh is run? I should note that we haven=92t modified = > alarmprog.sh, except for the two lines mentioned above. Since March 30, = > when we migrated to our new server, we have backed up 890 logical logs. = > This error has after 11 of the backups when = > alarmprog.sh/post_alarm_message/ph_dbs_alert was executed. =20 > > > From: Alexandre Marini <alexandre@briug.org> > > Subject: RE: SCHAPI post_alarm_messages [30138] > > Date: April 25, 2013 10:38:08 AM MST > > To: <ids@iiug.org> > > Reply-To: <ids@iiug.org> > >=20 > >=20 > > Hello.=20 > > Have you tried to rebuild the sysadmin after the upgrade? There were = > some=20 > > issues where the inplace upgrade did not upgrade sysadmin database, = > and you=20 > > have to do it manually as a workaround.=20 > >=20 > > (Backup any of your additional tasks, if you have some, before doing = > it, it=20 > > will be build with all defaults, only).=20 > >=20 > > Hope it helps.=20 > > Regards.=20 > >=20 > > Alexandre Marini=20 > > IBM Informix Certified Professional v10 / v11.50 / v11.70=20 > >=20 > > IBM Information Management Informix Technical Professional=20 > >=20 > > IBM Infosphere DataStage Technical Professional=20 > > Informix Senior DBA - Orizon Brasil=20 > > BRIUG website administrator=20 > > Informix independent consultant=20 > >=20 > >> To: ids@iiug.org=20 > >> From: JRUBIN@midwestern.edu=20 > >> Subject: SCHAPI post_alarm_messages [30137]=20 > >> Date: Thu, 25 Apr 2013 13:12:22 -0400=20 > >>=20 > >> I migrated from IDS 11.50 on HPUX to IDS 11.70.FC4 on SLES 11 SP 2 a = > few=20 > >> weeks ago. Since that time, I=B9m seeing periodic messages like this = > in the=20 > >> online logs:=20 > >>=20 > >> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not = > delete=20 > >> a row.=20 > >> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM error:=20= > > >> deadlock detected=20 > >>=20 > >> Is this related to something running in the scheduler? I do see this = > task=20 > >> from sysadmin:ph_task:=20 > >>=20 > >> tk_id 1=20 > >> tk_name m on_command_history=20 > >> tk_description Monitor how much data is kept in the command history = > table=20 > >> tk_type TASK=20 > >> tk_sequence 72=20 > >> tk_result_table=20 > >> tk_create=20 > >> tk_dbs sysadmin=20 > >> tk_execute delete from command_history where cmd_exec_time < (=20 > >>=20 > >> select current - value::INTERVAL DAY to SECOND=20 > >>=20 > >> from ph_threshold=20 > >>=20 > >> where name =3D 'COMMAND HISTORY RETENTION' )=20 > >> tk_delete 0 01:00:00=20 > >> tk_start_time 02:00:00=20 > >> tk_stop_time=20 > >> tk_frequency 1 00:00:00=20 > >> tk_next_execution 2013-04-26 09:54:51=20 > >> tk_total_executio+ 72=20 > >> tk_total_time 0.10489027422=20 > >>=20 > >> What is the best way to troubleshoot where this error is coming > from?=20= > > >>=20 > >> Here is the full history of this error over the past couple of > weeks:=20= > > >>=20 > >> 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -143 = > ISAM=20 > >> error: deadlock detected=20 > >>=20 > >> 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -143 = > ISAM=20 > >> error: deadlock detected=20 > >>=20 > >> 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -143 = > ISAM=20 > >> error: deadlock detected=20 > >>=20 > >> 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-12 01:04:36 SCHAPI: [post_alarm_message 15-972] Error -143 = > ISAM=20 > >> error: deadlock detected=20 > >>=20 > >> 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-19 15:00:10 SCHAPI: [post_alarm_message 15-1215] Error -143 = > ISAM=20 > >> error: deadlock detected=20 > >>=20 > >> 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-22 03:00:23 SCHAPI: [post_alarm_message 15-1294] Error -143 = > ISAM=20 > >> error: deadlock detected=20 > >>=20 > >> 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-22 16:00:21 SCHAPI: [post_alarm_message 15-1306] Error -143 = > ISAM=20 > >> error: deadlock detected=20 > >>=20 > >> 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 = > Could=20 > >> not delete a row.=20 > >> 2013-04-25 05:00:08 SCHAPI: [post_alarm_message 15-1

No, it isn=92t two logs being backed up close together. The delete = error just occurred today at 16:40:12 after logical log 890 filled and = was backed up. Log 889 filled and was backed up at 15:00:57. There = isn=92t much in the logs surrounding the 16:40:12 event: 16:30:20 Checkpoint Completed: duration was 0 seconds. 16:30:20 Wed May 8 - loguniq 890, logpos 0xacb6018, timestamp: = 0x8dc4ab53 Interval: 7675 16:30:20 Maximum server connections 688 16:30:20 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns = blocked 0, Plog used 970, Llog used 877 16:40:12 Logical Log 890 Complete, timestamp: 0x8dc8c8e3. 16:40:12 Logical Log 890 - Backup Started 16:40:12 Logical Log 890 - Backup Completed 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -240 Could not = delete a row. 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -143 ISAM error: = deadlock detected 16:45:20 Checkpoint Completed: duration was 0 seconds. 16:45:20 Wed May 8 - loguniq 891, logpos 0x2e07018, timestamp: = 0x8dcae3aa Interval: 7676 16:45:20 Maximum server connections 688 16:45:20 Checkpoint Statistics - Avg. Txn Block Time 0.000, # Txns = blocked 0, Plog used 2217, Llog used 20817 > From: Art Kagel <art.kagel@gmail.com> > Subject: Re: SCHAPI post_alarm_messages [30228] > Date: May 8, 2013 3:45:17 PM MST > To: <ids@iiug.org> > Reply-To: <ids@iiug.org> >=20 >=20 > Is it possible that at those times when the error has occurred two = logs=20 > completed very close on each other's heels? I also don't know what = would=20 > be being deleted, but it looks like you have a task scheduler sensor=20= > detecting ALARM events and it is something being done by that task=20 > scheduler procedure.=20 >=20 > Art=20 >=20 > Art S. Kagel=20 > Advanced DataTools (www.advancedatatools.com)=20 > Blog: http://informix-myview.blogspot.com/=20 >=20 > Disclaimer: Please keep in mind that my own opinions are my own = opinions=20 > and do not reflect on my employer, Advanced DataTools, the IIUG, nor = any=20 > other organization with which I am associated either explicitly,=20 > implicitly, or by inference. Neither do those opinions reflect those = of=20 > other individuals affiliated with any entity with which I am = affiliated nor=20 > those of the entities themselves.=20 >=20 > On Wed, May 8, 2013 at 6:36 PM, Jim Rubinstein = <jrubin@midwestern.edu>wrote:=20 >=20 >> I haven't tried to rebuild sysadmin yet, but I=3D92ve looked into = this a =3D=20 >> bit more. What is happening is that the error occurs after a logical = =3D=20 >> log is filled and is backed up. In alarmprog.sh, we=3D92ve set the =3D=20= >> following:=20 >>=20 >> BACKUPLOGS=3D3DY=20 >> BACKUP_CMD=3D3D"ontape -a -d"=20 >>=20 >> We=3D92ll go days backup logical logs as they fill, but on occasion = we=3D92ll =3D=20 >> get an error like this:=20 >>=20 >> 16:40:12 Logical Log 890 Complete, timestamp: 0x8dc8c8e3.=20 >> 16:40:12 Logical Log 890 - Backup Started=20 >> 16:40:12 Logical Log 890 - Backup Completed=20 >> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -240 Could not =3D=20= >> delete a row.=20 >>=20 >> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -143 ISAM error: = =3D=20 >> deadlock detected=20 >>=20 >> The error is with the post_alarm_message task, which executes the =3D=20= >> ph_dbs_alert command. I see the following routine in sysadmin:=20 >>=20 >> create function informix.ph_dbs_alert(informix.integer, =3D=20>> informix.integer,=20 >>=20 >> informix.pointer)=20 >>=20 >> returns integer=20 >>=20 >> external name '(ph_dbs_alert)'=20 >>=20 >> language C;=20 >>=20 >> What I haven't figured out is what the database is trying to delete. = Do =3D=20 >> you think a rebuild of sysadmin is still needed or is there a way to = =3D=20 >> better try to figure out what is happening as the logical logs fill = and =3D=20 >> alarmprog.sh is run? I should note that we haven=3D92t modified =3D=20= >> alarmprog.sh, except for the two lines mentioned above. Since March = 30, =3D=20 >> when we migrated to our new server, we have backed up 890 logical = logs. =3D=20 >> This error has after 11 of the backups when =3D=20 >> alarmprog.sh/post_alarm_message/ph_dbs_alert was executed. =3D20=20 >>=20 >>> From: Alexandre Marini <alexandre@briug.org>=20 >>> Subject: RE: SCHAPI post_alarm_messages [30138]=20 >>> Date: April 25, 2013 10:38:08 AM MST=20 >>> To: <ids@iiug.org>=20 >>> Reply-To: <ids@iiug.org>=20 >>> =3D20=20 >>> =3D20=20 >>> Hello.=3D20=20 >>> Have you tried to rebuild the sysadmin after the upgrade? There were = =3D=20 >> some=3D20=20 >>> issues where the inplace upgrade did not upgrade sysadmin database, = =3D=20 >> and you=3D20=20 >>> have to do it manually as a workaround.=3D20=20 >>> =3D20=20 >>> (Backup any of your additional tasks, if you have some, before doing = =3D=20 >> it, it=3D20=20 >>> will be build with all defaults, only).=3D20=20 >>> =3D20=20 >>> Hope it helps.=3D20=20 >>> Regards.=3D20=20 >>> =3D20=20 >>> Alexandre Marini=3D20=20 >>> IBM Informix Certified Professional v10 / v11.50 / v11.70=3D20=20 >>> =3D20=20 >>> IBM Information Management Informix Technical Professional=3D20=20 >>> =3D20=20 >>> IBM Infosphere DataStage Technical Professional=3D20=20 >>> Informix Senior DBA - Orizon Brasil=3D20=20 >>> BRIUG website administrator=3D20=20 >>> Informix independent consultant=3D20=20 >>> =3D20=20 >>>> To: ids@iiug.org=3D20=20 >>>> From: JRUBIN@midwestern.edu=3D20=20 >>>> Subject: SCHAPI post_alarm_messages [30137]=3D20=20 >>>> Date: Thu, 25 Apr 2013 13:12:22 -0400=3D20=20 >>>> =3D20=20 >>>> I migrated from IDS 11.50 on HPUX to IDS 11.70.FC4 on SLES 11 SP 2 = a =3D=20 >> few=3D20=20 >>>> weeks ago. Since that time, I=3DB9m seeing periodic messages like = this =3D=20 >> in the=3D20=20 >>>> online logs:=3D20=20 >>>> =3D20=20 >>>> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not = =3D=20 >> delete=3D20=20 >>>> a row.=3D20=20 >>>> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM = error:=3D20=3D=20 >>=20 >>>> deadlock detected=3D20=20 >>>> =3D20=20 >>>> Is this related to something running in the scheduler? I do see = this =3D=20 >> task=3D20=20 >>>> from sysadmin:ph_task:=3D20=20 >>>> =3D20=20 >>>> tk_id 1=3D20=20 >>>> tk_name m on_command_history=3D20=20 >>>> tk_description Monitor how much data is kept in the command history = =3D=20 >> table=3D20=20 >>>> tk_type TASK=3D20=20 >>>> tk_sequence 72=3D20=20 >>>> tk_result_table=3D20=20 >>>> tk_create=3D20=20 >>>> tk_dbs sysadmin=3D20=20 >>>> tk_execute delete from command_history where cmd_exec_time < (=3D20=20= >>>> =3D20=20 >>>> select current - value::INTERVAL DAY to SECOND=3D20=20 >>>> =3D20=20 >>>> from ph_threshold=3D20=20 >>>> =3D20=20 >>>> where name =3D3D 'COMMAND HISTORY RETENTION' )=3D20=20 >>>> tk_delete 0 01:00:00=3D20=20 >>>> tk_start_time 02:00:00=3D20=20 >>>> tk_stop_time=3D20=20 >>>> tk_frequency 1 00:00:0

I=92m not sure if inline images will show, but in OAT, the = post_alarm_message task looks like this: > From: Art Kagel <art.kagel@gmail.com> > Subject: Re: SCHAPI post_alarm_messages [30228] > Date: May 8, 2013 3:45:17 PM MST > To: <ids@iiug.org> > Reply-To: <ids@iiug.org> >=20 >=20 > Is it possible that at those times when the error has occurred two = logs=20 > completed very close on each other's heels? I also don't know what = would=20 > be being deleted, but it looks like you have a task scheduler sensor=20= > detecting ALARM events and it is something being done by that task=20 > scheduler procedure.=20 >=20 > Art=20 >=20 > Art S. Kagel=20 > Advanced DataTools (www.advancedatatools.com)=20 > Blog: http://informix-myview.blogspot.com/=20 >=20 > Disclaimer: Please keep in mind that my own opinions are my own = opinions=20 > and do not reflect on my employer, Advanced DataTools, the IIUG, nor = any=20 > other organization with which I am associated either explicitly,=20 > implicitly, or by inference. Neither do those opinions reflect those = of=20 > other individuals affiliated with any entity with which I am = affiliated nor=20 > those of the entities themselves.=20 >=20 > On Wed, May 8, 2013 at 6:36 PM, Jim Rubinstein = <jrubin@midwestern.edu>wrote:=20 >=20 >> I haven't tried to rebuild sysadmin yet, but I=3D92ve looked into = this a =3D=20 >> bit more. What is happening is that the error occurs after a logical = =3D=20 >> log is filled and is backed up. In alarmprog.sh, we=3D92ve set the =3D=20= >> following:=20 >>=20 >> BACKUPLOGS=3D3DY=20 >> BACKUP_CMD=3D3D"ontape -a -d"=20 >>=20 >> We=3D92ll go days backup logical logs as they fill, but on occasion = we=3D92ll =3D=20 >> get an error like this:=20 >>=20 >> 16:40:12 Logical Log 890 Complete, timestamp: 0x8dc8c8e3.=20 >> 16:40:12 Logical Log 890 - Backup Started=20 >> 16:40:12 Logical Log 890 - Backup Completed=20 >> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -240 Could not =3D=20= >> delete a row.=20 >>=20 >> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -143 ISAM error: = =3D=20 >> deadlock detected=20 >>=20 >> The error is with the post_alarm_message task, which executes the =3D=20= >> ph_dbs_alert command. I see the following routine in sysadmin:=20 >>=20 >> create function informix.ph_dbs_alert(informix.integer, =3D=20>> informix.integer,=20 >>=20 >> informix.pointer)=20 >>=20 >> returns integer=20 >>=20 >> external name '(ph_dbs_alert)'=20 >>=20 >> language C;=20 >>=20 >> What I haven't figured out is what the database is trying to delete. = Do =3D=20 >> you think a rebuild of sysadmin is still needed or is there a way to = =3D=20 >> better try to figure out what is happening as the logical logs fill = and =3D=20 >> alarmprog.sh is run? I should note that we haven=3D92t modified =3D=20= >> alarmprog.sh, except for the two lines mentioned above. Since March = 30, =3D=20 >> when we migrated to our new server, we have backed up 890 logical = logs. =3D=20 >> This error has after 11 of the backups when =3D=20 >> alarmprog.sh/post_alarm_message/ph_dbs_alert was executed. =3D20=20 >>=20 >>> From: Alexandre Marini <alexandre@briug.org>=20 >>> Subject: RE: SCHAPI post_alarm_messages [30138]=20 >>> Date: April 25, 2013 10:38:08 AM MST=20 >>> To: <ids@iiug.org>=20 >>> Reply-To: <ids@iiug.org>=20 >>> =3D20=20 >>> =3D20=20 >>> Hello.=3D20=20 >>> Have you tried to rebuild the sysadmin after the upgrade? There were = =3D=20 >> some=3D20=20 >>> issues where the inplace upgrade did not upgrade sysadmin database, = =3D=20 >> and you=3D20=20 >>> have to do it manually as a workaround.=3D20=20 >>> =3D20=20 >>> (Backup any of your additional tasks, if you have some, before doing = =3D=20 >> it, it=3D20=20 >>> will be build with all defaults, only).=3D20=20 >>> =3D20=20 >>> Hope it helps.=3D20=20 >>> Regards.=3D20=20 >>> =3D20=20 >>> Alexandre Marini=3D20=20 >>> IBM Informix Certified Professional v10 / v11.50 / v11.70=3D20=20 >>> =3D20=20 >>> IBM Information Management Informix Technical Professional=3D20=20 >>> =3D20=20 >>> IBM Infosphere DataStage Technical Professional=3D20=20 >>> Informix Senior DBA - Orizon Brasil=3D20=20 >>> BRIUG website administrator=3D20=20 >>> Informix independent consultant=3D20=20 >>> =3D20=20 >>>> To: ids@iiug.org=3D20=20 >>>> From: JRUBIN@midwestern.edu=3D20=20 >>>> Subject: SCHAPI post_alarm_messages [30137]=3D20=20 >>>> Date: Thu, 25 Apr 2013 13:12:22 -0400=3D20=20 >>>> =3D20=20 >>>> I migrated from IDS 11.50 on HPUX to IDS 11.70.FC4 on SLES 11 SP 2 = a =3D=20 >> few=3D20=20 >>>> weeks ago. Since that time, I=3DB9m seeing periodic messages like = this =3D=20 >> in the=3D20=20 >>>> online logs:=3D20=20 >>>> =3D20=20 >>>> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not = =3D=20 >> delete=3D20=20 >>>> a row.=3D20=20 >>>> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM = error:=3D20=3D=20 >>=20 >>>> deadlock detected=3D20=20 >>>> =3D20=20 >>>> Is this related to something running in the scheduler? I do see = this =3D=20 >> task=3D20=20 >>>> from sysadmin:ph_task:=3D20=20 >>>> =3D20=20 >>>> tk_id 1=3D20=20 >>>> tk_name m on_command_history=3D20=20 >>>> tk_description Monitor how much data is kept in the command history = =3D=20 >> table=3D20=20 >>>> tk_type TASK=3D20=20 >>>> tk_sequence 72=3D20=20 >>>> tk_result_table=3D20=20 >>>> tk_create=3D20=20 >>>> tk_dbs sysadmin=3D20=20 >>>> tk_execute delete from command_history where cmd_exec_time < (=3D20=20= >>>> =3D20=20 >>>> select current - value::INTERVAL DAY to SECOND=3D20=20 >>>> =3D20=20 >>>> from ph_threshold=3D20=20 >>>> =3D20=20 >>>> where name =3D3D 'COMMAND HISTORY RETENTION' )=3D20=20 >>>> tk_delete 0 01:00:00=3D20=20 >>>> tk_start_time 02:00:00=3D20=20 >>>> tk_stop_time=3D20=20 >>>> tk_frequency 1 00:00:00=3D20=20 >>>> tk_next_execution 2013-04-26 09:54:51=3D20=20 >>>> tk_total_executio+ 72=3D20=20 >>>> tk_total_time 0.10489027422=3D20=20 >>>> =3D20=20 >>>> What is the best way to troubleshoot where this error is coming=20 >> from?=3D20=3D=20 >>=20 >>>> =3D20=20 >>>> Here is the full history of this error over the past couple of=20 >> weeks:=3D20=3D=20 >>=20 >>>> =3D20=20 >>>> 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -240 = =3D=20 >> Could=3D20=20 >>>> not delete a row.=3D20=20 >>>> 2013-04-08 02:17:55 SCHAPI: [post_alarm_message 15-854] Error -143 = =3D=20 >> ISAM=3D20=20 >>>> error: deadlock detected=3D20=20 >>>> =3D20=20 >>>> 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -240 = =3D=20 >> Could=3D20=20 >>>> not delete a row.=3D20=20 >>>> 2013-04-10 13:00:18 SCHAPI: [post_alarm_message 15-928] Error -143 = =3D=20 >> ISAM=3D20=20 >>>> error: deadlock detected=3D20=20 >>>> =3D20=20 >>>> 2013-04-11 12:00:28 SCHAPI: [post_alarm_message 15-960] Error -240 = =3D=20 >> Could=3D20=20 >>>> not de

No images. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Wed, May 8, 2013 at 7:11 PM, Jim Rubinstein <jrubin@midwestern.edu>wrote: > I=92m not sure if inline images will show, but in OAT, the = > post_alarm_message task looks like this: > > > From: Art Kagel <art.kagel@gmail.com> > > Subject: Re: SCHAPI post_alarm_messages [30228] > > Date: May 8, 2013 3:45:17 PM MST > > To: <ids@iiug.org> > > Reply-To: <ids@iiug.org> > >=20 > >=20 > > Is it possible that at those times when the error has occurred two = > logs=20 > > completed very close on each other's heels? I also don't know what = > would=20 > > be being deleted, but it looks like you have a task scheduler sensor=20= > > > detecting ALARM events and it is something being done by that task=20 > > scheduler procedure.=20 > >=20 > > Art=20 > >=20 > > Art S. Kagel=20 > > Advanced DataTools (www.advancedatatools.com)=20 > > Blog: http://informix-myview.blogspot.com/=20 > >=20 > > Disclaimer: Please keep in mind that my own opinions are my own = > opinions=20 > > and do not reflect on my employer, Advanced DataTools, the IIUG, nor = > any=20 > > other organization with which I am associated either explicitly,=20 > > implicitly, or by inference. Neither do those opinions reflect those = > of=20 > > other individuals affiliated with any entity with which I am = > affiliated nor=20 > > those of the entities themselves.=20 > >=20 > > On Wed, May 8, 2013 at 6:36 PM, Jim Rubinstein = > <jrubin@midwestern.edu>wrote:=20 > >=20 > >> I haven't tried to rebuild sysadmin yet, but I=3D92ve looked into = > this a =3D=20 > >> bit more. What is happening is that the error occurs after a logical = > =3D=20 > >> log is filled and is backed up. In alarmprog.sh, we=3D92ve set the > =3D=20= > > >> following:=20 > >>=20 > >> BACKUPLOGS=3D3DY=20 > >> BACKUP_CMD=3D3D"ontape -a -d"=20 > >>=20 > >> We=3D92ll go days backup logical logs as they fill, but on occasion = > we=3D92ll =3D=20 > >> get an error like this:=20 > >>=20 > >> 16:40:12 Logical Log 890 Complete, timestamp: 0x8dc8c8e3.=20 > >> 16:40:12 Logical Log 890 - Backup Started=20 > >> 16:40:12 Logical Log 890 - Backup Completed=20 > >> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -240 Could not > =3D=20= > > >> delete a row.=20 > >>=20 > >> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -143 ISAM error: = > =3D=20 > >> deadlock detected=20 > >>=20 > >> The error is with the post_alarm_message task, which executes the > =3D=20= > > >> ph_dbs_alert command. I see the following routine in sysadmin:=20 > >>=20 > >> create function informix.ph_dbs_alert(informix.integer, =3D=20> >> informix.integer,=20 > >>=20 > >> informix.pointer)=20 > >>=20 > >> returns integer=20 > >>=20 > >> external name '(ph_dbs_alert)'=20 > >>=20 > >> language C;=20 > >>=20 > >> What I haven't figured out is what the database is trying to delete. = > Do =3D=20 > >> you think a rebuild of sysadmin is still needed or is there a way to = > =3D=20 > >> better try to figure out what is happening as the logical logs fill = > and =3D=20 > >> alarmprog.sh is run? I should note that we haven=3D92t modified =3D=20= > > >> alarmprog.sh, except for the two lines mentioned above. Since March = > 30, =3D=20 > >> when we migrated to our new server, we have backed up 890 logical = > logs. =3D=20 > >> This error has after 11 of the backups when =3D=20 > >> alarmprog.sh/post_alarm_message/ph_dbs_alert was executed. =3D20=20 > >>=20 > >>> From: Alexandre Marini <alexandre@briug.org>=20 > >>> Subject: RE: SCHAPI post_alarm_messages [30138]=20 > >>> Date: April 25, 2013 10:38:08 AM MST=20 > >>> To: <ids@iiug.org>=20 > >>> Reply-To: <ids@iiug.org>=20 > >>> =3D20=20 > >>> =3D20=20 > >>> Hello.=3D20=20 > >>> Have you tried to rebuild the sysadmin after the upgrade? There were = > =3D=20 > >> some=3D20=20 > >>> issues where the inplace upgrade did not upgrade sysadmin database, = > =3D=20 > >> and you=3D20=20 > >>> have to do it manually as a workaround.=3D20=20 > >>> =3D20=20 > >>> (Backup any of your additional tasks, if you have some, before doing = > =3D=20 > >> it, it=3D20=20 > >>> will be build with all defaults, only).=3D20=20 > >>> =3D20=20 > >>> Hope it helps.=3D20=20 > >>> Regards.=3D20=20 > >>> =3D20=20 > >>> Alexandre Marini=3D20=20 > >>> IBM Informix Certified Professional v10 / v11.50 / v11.70=3D20=20 > >>> =3D20=20 > >>> IBM Information Management Informix Technical Professional=3D20=20 > >>> =3D20=20 > >>> IBM Infosphere DataStage Technical Professional=3D20=20 > >>> Informix Senior DBA - Orizon Brasil=3D20=20 > >>> BRIUG website administrator=3D20=20 > >>> Informix independent consultant=3D20=20 > >>> =3D20=20 > >>>> To: ids@iiug.org=3D20=20 > >>>> From: JRUBIN@midwestern.edu=3D20=20 > >>>> Subject: SCHAPI post_alarm_messages [30137]=3D20=20 > >>>> Date: Thu, 25 Apr 2013 13:12:22 -0400=3D20=20 > >>>> =3D20=20 > >>>> I migrated from IDS 11.50 on HPUX to IDS 11.70.FC4 on SLES 11 SP 2 = > a =3D=20 > >> few=3D20=20 > >>>> weeks ago. Since that time, I=3DB9m seeing periodic messages like = > this =3D=20 > >> in the=3D20=20 > >>>> online logs:=3D20=20 > >>>> =3D20=20 > >>>> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -240 Could not = > =3D=20 > >> delete=3D20=20 > >>>> a row.=3D20=20 > >>>> 05:00:08 SCHAPI: [post_alarm_message 15-1391] Error -143 ISAM = > error:=3D20=3D=20 > >>=20 > >>>> deadlock detected=3D20=20 > >>>> =3D20=20 > >>>> Is this related to something running in the scheduler? I do see = > this =3D=20 > >> task=3D20=20 > >>>> from sysadmin:ph_task:=3D20=20 > >>>> =3D20=20 > >>>> tk_id 1=3D20=20 > >>>> tk_name m on_command_history=3D20=20 > >>>> tk_description Monitor how much data is kept in the command history = > =3D=20 > >> table=3D20=20 > >>>> tk_type TASK=3D20=20 > >>>> tk_sequence 72=3D20=20 > >>>> tk_result_table=3D20=20 > >>>> tk_create=3D20=20 > >>>> tk_dbs sysadmin=3D20=20 > >>>> tk_execute delete from command_history where cmd_exec_time < > (=3D20=20= > > >>>> =3D20=20 > >>>> select current - value::INTERVAL DAY to SECOND=3D20=20 > >>>> =3D20=20 > >>>> from ph_threshold=3D20=20 > >>>> =3D20=20 > >>>> where name =3D3D 'COMMAND HISTORY RETENTION' )=3D20=20 > >>>> tk_delete 0 01:00:00=3D20=20 > >>>> tk_start_time 02:00:00=3D20=20 > >>>> tk_stop_time=3D20=20 > >>>> tk_frequency 1 00:00:00=3D20=20 > >>>> tk_next_execution 2013-04-26 09:54:51=3D20=20 > >>>> tk_total_executio+ 72=3D20=20 > >>>> tk_total_time 0.10489027422=3D20=2

With a support call, we were able to figure out what was going on with = deadlock errors, with post_alarm_messages when logical logs fill and are = backed up. This is a know issue. Here=92s the response from Informix = support: --- APAR#: IC78848=20 CONCURRENT DELETE WITH BATCHEDREAD_INDEX SET ISSUES SPORADIC DEADLOCK = ERRORS=20 The problem is fixed in 11.70.FC5 and higher versions.=20 You have 2 options:=20 1. Upgrade your server to 11.70.FC5 or higher version.=20 2. There is a workaround to stop the error and it is to turn off = BATCHEDREAD_INDEX.=20 If you do not have the need for batched read index, you can turn = it off by setting=20 BATCHEDREAD_INDEX to 0 in the onconfig and the restart the ser.=20= Or, you can turn it off dynamically with onmode utility.=20 onmode -wf BATCHEDREAD_INDEX=3D0=20 =20 You will see this message on command line as well as in = the online.log=20 =20 Value of BATCHEDREAD_INDEX has been changed to 0.=20= =20 then, stop and restart the dbScheduler.=20 Connect to sysadmin database in dbaccess and run = these.=20 EXECUTE FUNCTION task('scheduler stop');=20 EXECUTE FUNCTION task('scheduler start');=20 --- Unfortunately, we obtain Informix from our ERP software provider and = .FC4 is the highest version they=92ve released and/or support. I am a = bit hesitant to disable BATCHEDREAD_INDEX, since the documentation = states =93this reduces the number of times that a buffer is read, thus = improving performance=94. I suppose I have a third option of just = living with the error and waiting for .FC5 or above, since it doesn=92t = appear to be hurting anything. =20 > From: Art Kagel <art.kagel@gmail.com> > Subject: Re: SCHAPI post_alarm_messages [30232] > Date: May 8, 2013 5:17:26 PM MST > To: <ids@iiug.org> > Reply-To: <ids@iiug.org> >=20 >=20 > No images.=20 >=20 > Art=20 >=20 > Art S. Kagel=20 > Advanced DataTools (www.advancedatatools.com)=20 > Blog: http://informix-myview.blogspot.com/=20 >=20 > Disclaimer: Please keep in mind that my own opinions are my own = opinions=20 > and do not reflect on my employer, Advanced DataTools, the IIUG, nor = any=20 > other organization with which I am associated either explicitly,=20 > implicitly, or by inference. Neither do those opinions reflect those = of=20 > other individuals affiliated with any entity with which I am = affiliated nor=20 > those of the entities themselves.=20 >=20 > On Wed, May 8, 2013 at 7:11 PM, Jim Rubinstein = <jrubin@midwestern.edu>wrote:=20 >=20 >> I=3D92m not sure if inline images will show, but in OAT, the =3D=20 >> post_alarm_message task looks like this:=20 >>=20 >>> From: Art Kagel <art.kagel@gmail.com>=20 >>> Subject: Re: SCHAPI post_alarm_messages [30228]=20 >>> Date: May 8, 2013 3:45:17 PM MST=20 >>> To: <ids@iiug.org>=20 >>> Reply-To: <ids@iiug.org>=20 >>> =3D20=20 >>> =3D20=20 >>> Is it possible that at those times when the error has occurred two =3D= =20 >> logs=3D20=20 >>> completed very close on each other's heels? I also don't know what =3D= =20 >> would=3D20=20 >>> be being deleted, but it looks like you have a task scheduler = sensor=3D20=3D=20 >>=20 >>> detecting ALARM events and it is something being done by that = task=3D20=20 >>> scheduler procedure.=3D20=20 >>> =3D20=20 >>> Art=3D20=20 >>> =3D20=20 >>> Art S. Kagel=3D20=20 >>> Advanced DataTools (www.advancedatatools.com)=3D20=20 >>> Blog: http://informix-myview.blogspot.com/=3D20=20 >>> =3D20=20 >>> Disclaimer: Please keep in mind that my own opinions are my own =3D=20= >> opinions=3D20=20 >>> and do not reflect on my employer, Advanced DataTools, the IIUG, nor = =3D=20 >> any=3D20=20 >>> other organization with which I am associated either explicitly,=3D20=20= >>> implicitly, or by inference. Neither do those opinions reflect those = =3D=20 >> of=3D20=20 >>> other individuals affiliated with any entity with which I am =3D=20 >> affiliated nor=3D20=20 >>> those of the entities themselves.=3D20=20 >>> =3D20=20 >>> On Wed, May 8, 2013 at 6:36 PM, Jim Rubinstein =3D=20 >> <jrubin@midwestern.edu>wrote:=3D20=20 >>> =3D20=20 >>>> I haven't tried to rebuild sysadmin yet, but I=3D3D92ve looked into = =3D=20 >> this a =3D3D=3D20=20 >>>> bit more. What is happening is that the error occurs after a = logical =3D=20 >> =3D3D=3D20=20 >>>> log is filled and is backed up. In alarmprog.sh, we=3D3D92ve set = the=20 >> =3D3D=3D20=3D=20 >>=20 >>>> following:=3D20=20 >>>> =3D20=20 >>>> BACKUPLOGS=3D3D3DY=3D20=20 >>>> BACKUP_CMD=3D3D3D"ontape -a -d"=3D20=20 >>>> =3D20=20 >>>> We=3D3D92ll go days backup logical logs as they fill, but on = occasion =3D=20 >> we=3D3D92ll =3D3D=3D20=20 >>>> get an error like this:=3D20=20 >>>> =3D20=20 >>>> 16:40:12 Logical Log 890 Complete, timestamp: 0x8dc8c8e3.=3D20=20 >>>> 16:40:12 Logical Log 890 - Backup Started=3D20=20 >>>> 16:40:12 Logical Log 890 - Backup Completed=3D20=20 >>>> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -240 Could not=20= >> =3D3D=3D20=3D=20 >>=20 >>>> delete a row.=3D20=20 >>>> =3D20=20 >>>> 16:40:12 SCHAPI: [post_alarm_message 15-1859] Error -143 ISAM = error: =3D=20 >> =3D3D=3D20=20 >>>> deadlock detected=3D20=20 >>>> =3D20=20 >>>> The error is with the post_alarm_message task, which executes the=20= >> =3D3D=3D20=3D=20 >>=20 >>>> ph_dbs_alert command. I see the following routine in sysadmin:=3D20=20= >>>> =3D20=20 >>>> create function informix.ph_dbs_alert(informix.integer, =3D3D=3D20=20= >>>> informix.integer,=3D20=20 >>>> =3D20=20 >>>> informix.pointer)=3D20=20 >>>> =3D20=20 >>>> returns integer=3D20=20 >>>> =3D20=20 >>>> external name '(ph_dbs_alert)'=3D20=20 >>>> =3D20=20 >>>> language C;=3D20=20 >>>> =3D20=20 >>>> What I haven't figured out is what the database is trying to = delete. =3D=20 >> Do =3D3D=3D20=20 >>>> you think a rebuild of sysadmin is still needed or is there a way = to =3D=20 >> =3D3D=3D20=20 >>>> better try to figure out what is happening as the logical logs fill = =3D=20 >> and =3D3D=3D20=20 >>>> alarmprog.sh is run? I should note that we haven=3D3D92t modified = =3D3D=3D20=3D=20 >>=20 >>>> alarmprog.sh, except for the two lines mentioned above. Since March = =3D=20 >> 30, =3D3D=3D20=20 >>>> when we migrated to our new server, we have backed up 890 logical =3D= =20 >> logs. =3D3D=3D20=20 >>>> This error has after 11 of the backups when =3D3D=3D20=20 >>>> alarmprog.sh/post_alarm_message/ph_dbs_alert was executed. =3D3D20=3D= 20=20 >>>> =3D20=20 >>>>> From: Alexandre Marini <alexandre@briug.org>=3D20=20 >>>>> Subject: RE: SCHAPI post_alarm_messages [30138]=3D20=20 >>>>> Date: April 25, 2013 10:38:08 AM MST=3D20=20 >>>>> To: <ids@iiug.org>=3D20=20 >>>>> Reply-To: <ids@iiug.org>=3D20=20 >>>>> =3D3D20=3D20=20 >>>>> =3D3D20=3D20=20 >>>>> Hello.=3D3D20=3D20=20 >>>>> Have you tried to rebuild the sysadmin after the upgrade? There = wer