Dirty page queues during a checkpoint (WAS: The old raw devices chestnut)
Posted in 2004
Topics: Performance & Tuning, Storage & Space Management, Logging & Checkpoints
This horse is nearly dead now, but I do have one more question, if i
may:
When pages are queued for flushing to disk during a checkpoint, how is
it queued? Is there a queue per chunk, or is there a queue per dbspace,
or am i completely wrong here? And may this affect performance?
The reason i ask is because i've noticed that I/O sometimes peters out
as the checkpoint progresses. I know this because i run onstat -P every
so many seconds and sometimes you can see later in the checkpoint that
there are dirty pages outstanding for one large table only (which has
its own dbspace). Has anybody seen this (apparent) behaviour?
Then i must say that I do hate this checkpoint business. Or beter, i
hate the fact that Informix blocks during a checkpoint. This particular
feature of Informix has elevated the art of checkpoint duration tuning
to something close to black magic. I can't think of any other dbms's
that do that - blocking i mean.
Lastly, i think with the (very fast) growing trend towards
virtualization of storage (on the hardware and software layers) that
fs-vs-raw should eventually become a moot issue. I mean, at one point in
time we ran our database off a Netapp filer (and therefore on NFS), and
what a brilliant piece of storage virtualization that is!
Just my 6 cents.
-------------------------------------------
Willem Roos - (+27) 21 980 4941
Per sercas vi malkovri
> -----Original Message-----
> Removed for brevity
sending to informix-list
They're queued in LRU queues as they become dirty, and then during the
checkpoint each chunk with stuff to be written gets assigned to a page
cleaner and within the chunk the pages are sorted into sector order
prior to writing. So it's highly possible that not all the chunks (and
therefore not all the tables) will complete at once.
You can watch it all happen with onstat -F too.
Andy
"Willem Roos" <wroos@shoprite.co.za> wrote in message news:<c5obgs$beh$1@terabinaries.xmission.com>...
> This horse is nearly dead now, but I do have one more question, if i
> may:
>
> When pages are queued for flushing to disk during a checkpoint, how is
> it queued? Is there a queue per chunk, or is there a queue per dbspace,
> or am i completely wrong here? And may this affect performance?
>
> The reason i ask is because i've noticed that I/O sometimes peters out
> as the checkpoint progresses. I know this because i run onstat -P every
> so many seconds and sometimes you can see later in the checkpoint that
> there are dirty pages outstanding for one large table only (which has
> its own dbspace). Has anybody seen this (apparent) behaviour?
>
> Then i must say that I do hate this checkpoint business. Or beter, i
> hate the fact that Informix blocks during a checkpoint. This particular
> feature of Informix has elevated the art of checkpoint duration tuning
> to something close to black magic. I can't think of any other dbms's
> that do that - blocking i mean.
>
> Lastly, i think with the (very fast) growing trend towards
> virtualization of storage (on the hardware and software layers) that
> fs-vs-raw should eventually become a moot issue. I mean, at one point in
> time we ran our database off a Netapp filer (and therefore on NFS), and
> what a brilliant piece of storage virtualization that is!
>
> Just my 6 cents.
>
> -------------------------------------------
> Willem Roos - (+27) 21 980 4941
> Per sercas vi malkovri
>
> > -----Original Message-----
> > Removed for brevity
>
> sending to informix-list