RE: Sudden Increase in Checkpoint Lengths
Keith.
Keith wrote:
> Is there any overhead
> with having this turned on (apart from more entries in the online log)
> and is it a feature available in IDS9.4?
We have it turned on in a high production environment and don't feel any
overhead. And yes it is available in IDS 9.4.
If you would like to check the btscanner:
onstat -C (Btree Cleaner Info BT scanner profile Information). If you haveonly one btscannner and only leaf scans that could be critical because your
indexes cannot be cleaned in time.
To change while IDS is running (starts two btscanners of type range):
onmode -C rangesize 500
onmode -C threshold 50000
onmode -C stop
onmode -C stop
onmode -C start
onmode -C start
Regards,
Reinhard.
> -----Original Message-----
> From: informix-list-bounces@iiug.org
> [mailto:informix-list-bounces@iiug.org]On Behalf Of Keith Simmons
> Sent: Tuesday, August 11, 2009 10:18 AM
> To: informix-list@iiug.org
> Subject: Re: Sudden Increase in Checkpoint Lengths
>
>
> Answers to All responses below:-
>
> 2009/8/11 Habichtsberg, Reinhard <RHabichtsberg@arz-emmendingen.de>:
> > Keith.
> >
> > If you haven't done it already:
> > set environment TRACECKPT=1; export TRACECKPT before starting IDS.> > That will make the checkpoints more chatty in online.log.
> >
> > If they are in critical section most of the time it could
> be an issue with
> > the btscanners.
> >
> > HTH.
> > Reinhard.
> >
> >> -----Original Message-----
> >> From: informix-list-bounces@iiug.org
> >> [mailto:informix-list-bounces@iiug.org]On Behalf Of John Carlson
> >> Sent: Tuesday, August 11, 2009 3:04 AM
> >> To: informix-list@iiug.org
> >> Subject: Re: Sudden Increase in Checkpoint Lengths
> >>
> >>
> >> Keith Simmons wrote:
> >> > All
> >> >
> >> > Running IDS 9.4 FC6 on AIX 5.3, application written in
> INFORMIX-4GL
> >> > Version 7.32.FC3.
> >> > (I know it's a bit old, but it runs well (till now !!))
> >> > Around Friday lunchtime the checkpoint times on this
> server jumped
> >> > from between 0 and 1 second to between 30 and 60 seconds
> (although
> >> > many are running arounf 6 to 10 seconds, big variances).
> Even though
> >> > the checkpoints are fuzzy (with several thousand buffers
> not being
> >> > flushed) the users are being blocked for this length of
> time every 5
> >> > minutes (and they are not happy !!!). All disk writes are
> >> Chunk Writes
> >> > (no LRU or Fg), buffers are only getting around 6% dirty between
> >> > checkpoints with low and high set to 8%/15%. Capturing onstat -u
> >> > during a checkpoint shows around 30 user sessions blocked on the
> >> > checkpoint but no-one in a critical section
> >> > Except for an engine bounce (planned) about 4 weeks ago,
> the engine
> >> > has run continuously with no configuration change for over
> >> 9 months.
> >> > There have been no application changes for the past 7 days.
> >> > Only clue to the long checkpoints was a couple of 'Failed
> >> _aioreturn'
> >> > errors on each of two separate (unrelated) chunks in the
> online log.
> >> > The O/S logs shows these errors also, but only as
> isolated entries,
> >> > nothing continuous or on-going, also there are no hardware error
> >> > lights illuminated nor does the DS4000 monitoring
> software show any
> >> > issues.
> >> > I have run page checks on the chunks with no errors and
> am currently
> >> > running -cxI and -cd on each table in the DBSpaces involved.
> >> > Any suggestions where else to look?
> >> >
> >>
> >> Any issues with disk in the syslog (or AIX's version of syslog)?
> >>
> >> JWC
> >> _______________________________________________
> >> Informix-list mailing list
> >> Informix-list@iiug.org
> >> http://www.iiug.org/mailman/listinfo/informix-list
> >>
> > _______________________________________________
> > Informix-list mailing list
> > Informix-list@iiug.org
> > http://www.iiug.org/mailman/listinfo/informix-list
> >
>
> Neil
> sar 5 5 while a checkpoint was in progress:-
> 09:07:30 device %busy avque r+w/s Kbs/s> avwait avserv
> Average hdisk0 2 0.0 7 34
> 3.8 4.0
> hdisk1 2 0.0 7 34
> 3.5 3.9
> dac0 0 0.0 173 968
> 39.6 10.6
> dac0-utm 0 0.0 0 0
> 0.0 0.0
> dac1 0 0.0 165 678
> 114.9 2.4
> dac1-utm 0 0.0 0 0
> 0.0 0.0
> hdisk2 0 0.0 0 1
> 0.0 6.2
> hdisk3 0 0.0 0 0
> 0.0 1.6
> hdisk4 0 0.0 0 0
> 0.0 0.0
> hdisk5 38 0.6 105 667
> 51.8 13.3
> hdisk6 0 0.0 2 15
> 0.0 5.1
> hdisk7 0 0.0 0 0
> 0.0 3.1
> hdisk8 0 1.7 43 180
> 385.8 0.8
> hdisk9 3 0.0 62 279
> 0.0 1.5
> hdisk10 0 0.0 3 14
> 0.0 5.5
> hdisk11 38 0.0 112 456
> 0.0 2.4
> hdisk12 0 0.0 0 0
> 0.0 0.0
> hdisk13 0 0.0 0 0
> 0.0 0.0
> hdisk14 0 0.0 0 0
> 0.0 0.0
> hdisk15 0 0.0 0 4
> 0.0 3.4
> hdisk16 0 0.0 0 0
> 0.0 2.7
> hdisk17 0 0.0 0 0
> 0.0 0.0
> hdisk18 2 0.0 5 21
> 0.0 4.5
> hdisk19 0 0.0 0 0
> 0.0 2.8
> hdisk20 0 0.0 0 0
> 0.0 0.1
> cd0 0 0.0 0 0
> 0.0 0.0
>
> Obnoxio
> No, data-set has not grown (I have just finished a round of purging
> data, didn't recover the disk space but this finished 3 weeks ago and
> the issue only appeared Friday). I only have one 'set' of disks, all
> same size, shape, speed and connected in the same way.
>
> Ian
> Yes, same place, same package but only two memory segments, Resident
> and 1 Virtual, no change from last restart.
>
> John
> Subsequent to my initial post I have had another LVM_IO_FAIL in the
> AIX errpt log. I'm just about to raise call re hardware (and possibly
> LVM).
>
> Reinhard
> Not possible currently as the application is 24 x 7. It is something I
> will consider next time I bounce the engine. Is there any overhead
> with having this turned on (apart from more entries in the online log)
> and is it a feature available in IDS9.4?
>
> All
> Archive (using ontape) times have increased by 1 hour (37 %) from
> before to after Friday midday. This is pointing more and more at
> hardware or O/S, calls are being raised.
>
> Ma