Re: Never ending Checkpoint
Habichtsberg, Reinhard wrote:
> IDS 9.40.FC4W2
> SunOS 5.8
>
> Last Saturday we had a "never ending checkpoint". There are no loggings
> about the event except the lines in online.log:
>
...
> 11:42:16 Logical Log 1497251 - Backup Started
> 13:45:54 Process exited with return code 137: /bin/sh /bin/sh -c
> /opt/lib/informix/etc/log_full.sh 2 23 "L> ogical Log 1497262 Complete, timestamp: 0x603a11fd." "Logical
> 14:26:23 WARNING: Checkpoint appears stalled and may not complete> before the database server shuts down.
>
> 14:31:51 IBM Informix Dynamic Server Started.>
> At 14:26:23 the server was shut down an onmode -ky and then startet with
> oninit. The "checkpoint request" lasted nearly three hours yet.
>
> I tried to find helpful advices in c.d.i about long lasting checkpoints and
> return code 137 but couldn't find anything that gives me an answer.
>
> Thanks in advance,
> Reinhard.
>
It is too late to say anything about the nature of the issue as IDS
server has been restarted w/o getting diagnostics.
Next time you observe a hung checkpoint run the command
%> $INFORMIXDIR/etc/evidence.sh 3 XX af.hung 0 x 0 x 1176
as user informix on the command line.
That shall generate an af.hung file that captures most of the important
information. Also a shmem dump will get generated. which might be useful
for IDS support.
Generally hung checkpoints have two possible reasons:
- IDS bug , when some lockjed resources such as mutexes cause a deadlock
between various threads so the chekpoint can never happen .
- IDS waits for the IO subsystem to resturn for the checkpoint to complete.
Short term solution is always an IDS restart .
HTH
Tilman