RE: Reducing onload time
> -----Original Message-----
> From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On
> Behalf Of Andreas.KUTSCHE@spar.at
> Sent: Tuesday, September 25, 2007 12:26 AM
> To: ids@iiug.org
> Subject: AW: Reducing onload time [9996]
>
> Hi,
>
> hardware data would be helpful (at least number of CPUs,
> available RAM, OS,
> disksystem
> with/without cache) - load times depend on hardware :-)
>
> If your are running without KAIO (kernel asynchronous I/O)
> you might increase
> NUMAIOVPS> further (check with onstat -g iov - as long as io/wup shows
> values > 1 for all
> AIO-vps you should increase NUMAIOVPS). Too many NUMAIOVPS
> don't hurt unless
> you are
> running on a very very small machine.
>
> A bigger PHYSFILE would help too - it reduces the number of
> checkpoints
> necessary.
>
> Is NUMCPUVPS appropriate for the machine?
>
> Check with onstat -F if you see Fg (Foreground) Writes - if
> yes, you MUST
> increase
> BUFFERS. If you have the memory available increasing BUFFERS
> will never hurt.
>
> Regards,
> Andreas Kutsche
>
<< snipped >>
> -----Ursprüngliche Nachricht-----
>
> > Von: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org]Im
> Auftrag von
> > PAUL RIDDING
> > Gesendet: Dienstag, 25. September 2007 02:29
> > An: ids@iiug.org
> > Betreff: Reducing onload time [9993]
> >
> >
> > We have a few test databases that we refresh periodically using
> > onunload/onload. The onload generally takes a very long time.
> > For example,
> > re-creating an 8Gb database takes approximately 8 hours!
> > Below are the details
> >
> > Onload command: "onload -t /dumpdir/dbfile -b 64 -s 10000000
> > -d tabldbs
> > testDB"
> >
> > In an attempt to reduce the load times I have changed the
> following:
> >
> > Changed PHYSFILE: 1000 => 20000
> > Changed CKPTINTVL: 900 => 3600
> > Changed NUMAIOVPS: 2 => 8
> > Changed BUFFERS: 20000 => 200000
> > Changed LRUS: 8 => 60
> > Changed LRU_MAX_DIRTY: 8 => 80
> > Changed LRU_MIN_DIRTY: 4 => 70
> > Changed RA_PAGES: => 32
> > Changed RA_THRESHOLD: => 16
> >
> > The above changes reduced the load time from about 12 hours
> > to 8 hours but I
> > think 8 hours is still excessive for a reasonably small DB.
> >
> > The main issue seems to be that during the load the engine is
> > checkpointing
> > excessively. Below is an extract of the online log: -
> >
<< snipped >>
If there is no one else accessing the db during the load, you might try
switching the database to non-logging for the duration of the load, then
switch it back to its normal logging status (Buffered or Unbuffered)
afterwards, followed immediately by a full level-0 archive.
HTH,
Paul M.