increase chunk size
After upgrading from IDS 7 to IDS 10.00FC6 on HP-UX, a user with 13 disks and 250 chunks asked whether to consolidate into fewer, larger chunks now that the 2GB limit is gone, and whether any sizing rule of thumb exists. Replies: no fixed rule — chunk size is mainly a manageability/max-size issue, though some favour smaller chunks for maximum performance. Key caveat raised: IDS assigns one page cleaner per chunk, so in 9.4/10.00 consolidating many dirty pages into one big chunk can lengthen blocking checkpoints (not an issue in v11's non-blocking checkpoints); if dirty pages already cluster in one chunk, size makes no difference. Also advised enabling expanded (big) chunk capacity after an upgrade to avoid needless page-structure conversion.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Hi, We upgraded IDS-7 to IDS-10.00FC6 in HP-UX. The chunk size used to be 2GB, but now in the IDS-10.00 it can be up to 4TB. We have 13 disks and 250 chunks allocated in the database. The question is should we increase the chunk size and decrease the number of chunks? Is there any rule of thumb that we can determine the new chunk size for 13 disks? Thanks a lot
TRI TRINH wrote: > Hi, > > We upgraded IDS-7 to IDS-10.00FC6 in HP-UX. The chunk size used to be 2GB, but > now in the IDS-10.00 it can be up to 4TB. We have 13 disks and 250 chunks > allocated in the database. The question is should we increase the chunk size > and decrease the number of chunks? Is there any rule of thumb that we can > determine the new chunk size for 13 disks? > Chunk sizing does not affect server performance in any substantive way, it's purely an ease of management and size of the maximum database issue. Size the chunks in any way that's convenient. Art S. Kagel Oninit > Thanks a lot > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
> TRI TRINH wrote: > > Hi, > > > > We upgraded IDS-7 to IDS-10.00FC6 in HP-UX. The chunk size used to be > 2GB, > but > > now in the IDS-10.00 it can be up to 4TB. We have 13 disks and 250 > chunks > > allocated in the database. The question is should we increase the > chunk size > > and decrease the number of chunks? Is there any rule of thumb that we > can > > determine the new chunk size for 13 disks? > > > > Chunk sizing does not affect server performance in any substantive way, > it's purely an ease of management and size of the maximum database > issue. Size the chunks in any way that's convenient. > > Art S. Kagel > Oninit This was a question asked at the Vienna conference at the ask the experts panel. The consensus was smaller chunks sizes were better if you want to get the last bit of performance out of the system. Paul Watson Tel: +1 913-400-2620 Mob: +1 913-387-7529 Web: www.oninit.com Failure is not as frightening as regret. If you want to improve, be content to be thought foolish and stupid. ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
I would suggest though to make sure that expanded chunk capacity is set to always, even if you do not use it. The reason to enable it, even if you aren't going to use it is because everything in memory uses the Big Chunk page structure, and while the conversion is not expensive, why use the CPU cycles for that if you do not have to? If this was a new 10 instance, nothing to worry about, but as an upgrade, the odds are that Big Chunks are disabled. FWIW ----- Original Message ---- From: Paul Watson <paul@oninit.com> To: ids@iiug.org Sent: Wednesday, January 23, 2008 3:46:17 PM Subject: RE: increase chunk size [11082] > TRI TRINH wrote: > > Hi, > > > > We upgraded IDS-7 to IDS-10.00FC6 in HP-UX. The chunk size used to be > 2GB, > but > > now in the IDS-10.00 it can be up to 4TB. We have 13 disks and 250 > chunks > > allocated in the database. The question is should we increase the > chunk size > > and decrease the number of chunks? Is there any rule of thumb that we > can > > determine the new chunk size for 13 disks? > > > > Chunk sizing does not affect server performance in any substantive way, > it's purely an ease of management and size of the maximum database > issue. Size the chunks in any way that's convenient. > > Art S. Kagel > Oninit This was a question asked at the Vienna conference at the ask the experts panel. The consensus was smaller chunks sizes were better if you want to get the last bit of performance out of the system. Paul Watson Tel: +1 913-400-2620 Mob: +1 913-387-7529 Web: www.oninit.com Failure is not as frightening as regret. If you want to improve, be content to be thought foolish and stupid. ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ =========== ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum. See you at the IIUG Informix 2008 Conference The Power Conference for Informix Professionals April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas http://www.iiug.org/conf Registration Now Open!!
In general, I would agree with Art, but the one place large chunks can currently hurt perform is at a checkpoint in version 9.4 and 10.00. Th= is is NOT true is version 11 due to non-blocking checkpoint. The scenario is based on the knowledge that IDS assigns one page cleaner (flusher) to each chunk. If all your modifications are to a single chunk, and those update do not qualify as fuzzy operations then your checkpoint times could increase. Now if you update are spread across several chunks this would not apply, but if you create a new instance with a single large chunk then please be aware of this limitation until version 11. John = "Art S. Kagel = (Oninit LLC)" = <art@oninit.com> = To Sent by: ids@iiug.org = ids-bounces@iiug. = cc org = Subj= ect Re: increase chunk size [11081]= 01/23/2008 01:38 = PM = = = Please respond to = ids@iiug.org = = = TRI TRINH wrote: > Hi, > > We upgraded IDS-7 to IDS-10.00FC6 in HP-UX. The chunk size used to be= 2GB, but > now in the IDS-10.00 it can be up to 4TB. We have 13 disks and 250 ch= unks > allocated in the database. The question is should we increase the chu= nk size > and decrease the number of chunks? Is there any rule of thumb that we= can > determine the new chunk size for 13 disks? > Chunk sizing does not affect server performance in any substantive way,= it's purely an ease of management and size of the maximum database issue. Size the chunks in any way that's convenient. Art S. Kagel Oninit > Thanks a lot > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D ***********************************************************************= ******** Forum Note: Use "Reply" to post a response in the discussion forum. See you at the IIUG Informix 2008 Conference The Power Conference for Informix Professionals April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas http://www.iiug.org/conf Registration Now Open!! =
John Miller iii wrote: > In general, I would agree with Art, but the one place large chunks can > currently hurt perform is at a checkpoint in version 9.4 and 10.00. Th= > is > is NOT true is version 11 due to non-blocking checkpoint. > John's right of course. I had just pointed that same thing out to someone else a few days ago and missed mentioning it in this reply. However, John's answer is also too simplistic. Strictly speaking it's not the the size of the chunks that may slow down the checkpoints, but the consolidation from what might otherwise have been many smaller chunks into one or a few larger ones. IFF there are many dirty pages and they would have spanned multiple chunks, then having the dirty pages all affecting a single chunk would degrade checkpoint processing, true and no argument. On the other hand, if all of the dirty pages for a dbspace would have affected rows in a single chunk before, it doesn't matter whether the chunk is large or small, the checkpoint duration will be the same. So if your working set is small and updated pages will tend to all cluster in the most recently added pages, which will tend to always be on only one or at most two chunks, then the chunk size will not affect the checkpoint duration in any material way. Since that's a typical OLTP environment, I have to stand on my original statement with the caviat that you have to know your system in order to make a proper determination in this matter. Art S. Kagel > The scenario is based on the knowledge that IDS assigns one page > cleaner (flusher) to each chunk. If all your modifications are to a > single chunk, and those update do not qualify as fuzzy operations > then your checkpoint times could increase. > > Now if you update are spread across several chunks this would not > apply, but if you create a new instance with a single large chunk > then please be aware of this limitation until version 11. > > John > > = > > "Art S. Kagel = > > (Oninit LLC)" = > > <art@oninit.com> = > To > > Sent by: ids@iiug.org = > > ids-bounces@iiug. = > cc > > org = > > Subj= > ect > > Re: increase chunk size [11081]= > > 01/23/2008 01:38 = > > PM = > > = > > = > > Please respond to = > > ids@iiug.org = > > = > > = > > TRI TRINH wrote: > >> Hi, >> >> We upgraded IDS-7 to IDS-10.00FC6 in HP-UX. The chunk size used to be= >> > > 2GB, > but > >> now in the IDS-10.00 it can be up to 4TB. We have 13 disks and 250 ch= >> > unks > > >> allocated in the database. The question is should we increase the chu= >> > nk > size > >> and decrease the number of chunks? Is there any rule of thumb that we= >> > can > > >> determine the new chunk size for 13 disks? >> >> > > Chunk sizing does not affect server performance in any substantive way,= > > it's purely an ease of management and size of the maximum database > issue. Size the chunks in any way that's convenient. > > Art S. Kagel > Oninit > > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========