tempdbs in raw or in filesystem

I have two temp dbspaces on same disk. One is using like raw devices with direct I/O and another like file in Linux file system. Both of them have almost same number of read/write but, using onstat -g iof command for monitoring system, looks like temp dbspace in file system is 100 times faster than raw dbspace ? Is this bug or this is because of Linux cache ? If it tempdbs on file system is really 100 times faster (when is talking about of small amount of data to read and write, but many many times), then it is good to use temp dbspace in file system instead in raw devices . Here it is, what onstat -g iof says: IBM Informix Dynamic Server Version 11.50.FC4WE ... AIO global files: gfd pathname bytes read page reads bytes write page writes io/s ... 5 /dev/raw3 6542811136 3194732 6865305600 3352204 125.1 op type count avg. time seeks 0 N/A reads 0 N/A writes 109 0.0005 kaio_reads 119218 0.0034 kaio_writes 141296 0.0119 6 tempdbs2 6508036096 3177752 6859382784 3349308 15970.2 op type count avg. time seeks 0 N/A reads 117239 0.0000 writes 137252 0.0001 kaio_reads 0 N/A kaio_writes 0 N/A

First, you are not using DIRECT_IO with the RAW device, it is not necessary as RAW doesn't go through the linux buffer cache anyway. In addition, note that you are not using KAIO for the tempdbs2 file so it is not using DIRECT_IO either. That means that, yes, you are seeing the effect of rereading the Linux buffer cache over and over. IDS writes to filesystem chunks using either DIRECT_IO or O_SYNC so that the buffer linux cache pages are either bypassed or flushed synchronously. Writes under DIRECT_IO are about 5% slower than to RAW in general and under O_SYNC they are as much as 25% slower than RAW in general. However, for temp tables that are written once and reread many times, there can be benefits. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) IIUG Board of Directors (art@iiug.org) See you at the 2010 IIUG Informix Conference April 25-28, 2010 Overland Park (Kansas City), KS www.iiug.org/conf Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Thu, May 6, 2010 at 11:51 AM, IVAN ZAVIS <ivan.zavis@mi-system.co.rs>wrote: > I have two temp dbspaces on same disk. One is using like raw devices with > direct I/O and another like file in Linux file system. Both of them have > almost same number of read/write but, using onstat -g iof command for > monitoring system, looks like temp dbspace in file system is 100 times > faster > than raw dbspace ? > > Is this bug or this is because of Linux cache ? > > If it tempdbs on file system is really 100 times faster (when is talking > about > of small amount of data to read and write, but many many times), then it is > good to use temp dbspace in file system instead in raw devices . > > Here it is, what onstat -g iof says: > > IBM Informix Dynamic Server Version 11.50.FC4WE ... > > AIO global files: > gfd pathname bytes read page reads bytes write page writes io/s > .... > 5 /dev/raw3 6542811136 3194732 6865305600 3352204 125.1 > > op type count avg. time > > seeks 0 N/A > > reads 0 N/A > > writes 109 0.0005 > > kaio_reads 119218 0.0034 > > kaio_writes 141296 0.0119 > > 6 tempdbs2 6508036096 3177752 6859382784 3349308 15970.2 > > op type count avg. time > > seeks 0 N/A > > reads 117239 0.0000 > > writes 137252 0.0001 > > kaio_reads 0 N/A > > kaio_writes 0 N/A > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --001636e0a66c16cce50485f0029c

Aha, I figured out (I think) ... DIRECT_IO parameter in onconfig says: Enables (1) or disables (0) direct I/O for chunks is using with ONLY for cooked files and only if that cooked file is NOT using for tempdbs .... I'm I right ? Thanks, Ivan. On 06.05.2010 19:09, Art Kagel wrote: > First, you are not using DIRECT_IO with the RAW device, it is not necessary > as RAW doesn't go through the linux buffer cache anyway. In addition, note > that you are not using KAIO for the tempdbs2 file so it is not using > DIRECT_IO either. That means that, yes, you are seeing the effect of > rereading the Linux buffer cache over and over. IDS writes to filesystem > chunks using either DIRECT_IO or O_SYNC so that the buffer linux cache pages > are either bypassed or flushed synchronously. Writes under DIRECT_IO are > about 5% slower than to RAW in general and under O_SYNC they are as much as > 25% slower than RAW in general. However, for temp tables that are written > once and reread many times, there can be benefits. > > Art > > Art S. Kagel > Advanced DataTools (www.advancedatatools.com) > IIUG Board of Directors (art@iiug.org) > > See you at the 2010 IIUG Informix Conference > April 25-28, 2010 > Overland Park (Kansas City), KS > www.iiug.org/conf > > Disclaimer: Please keep in mind that my own opinions are my own opinions and > do not reflect on my employer, Advanced DataTools, the IIUG, nor any other > organization with which I am associated either explicitly, implicitly, or by > inference. Neither do those opinions reflect those of other individuals > affiliated with any entity with which I am affiliated nor those of the > entities themselves. > > On Thu, May 6, 2010 at 11:51 AM, IVAN ZAVIS<ivan.zavis@mi-system.co.rs>wrote: > > >> I have two temp dbspaces on same disk. One is using like raw devices with >> direct I/O and another like file in Linux file system. Both of them have >> almost same number of read/write but, using onstat -g iof command for >> monitoring system, looks like temp dbspace in file system is 100 times >> faster >> than raw dbspace ? >> >> Is this bug or this is because of Linux cache ? >> >> If it tempdbs on file system is really 100 times faster (when is talking >> about >> of small amount of data to read and write, but many many times), then it is >> good to use temp dbspace in file system instead in raw devices . >> >> Here it is, what onstat -g iof says: >> >> IBM Informix Dynamic Server Version 11.50.FC4WE ... >> >> AIO global files: >> gfd pathname bytes read page reads bytes write page writes io/s >> .... >> 5 /dev/raw3 6542811136 3194732 6865305600 3352204 125.1 >> >> op type count avg. time >> >> seeks 0 N/A >> >> reads 0 N/A >> >> writes 109 0.0005 >> >> kaio_reads 119218 0.0034 >> >> kaio_writes 141296 0.0119 >> >> 6 tempdbs2 6508036096 3177752 6859382784 3349308 15970.2 >> >> op type count avg. time >> >> seeks 0 N/A >> >> reads 117239 0.0000 >> >> writes 137252 0.0001 >> >> kaio_reads 0 N/A >> >> kaio_writes 0 N/A >> >> >> >> >> > ******************************************************************************* > >> Forum Note: Use "Reply" to post a response in the discussion forum. >> >> >> > --001636e0a66c16cce50485f0029c > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Ivan Zaviç System& DB Administrator Mobile:+381-64-846-99-08 mail: ivan.zavis@mi-system.co.rs _________________________________________________________ M&I SYSTEMS CO. Cirila i Metodija 13/a, 21000 Novi Sad, Serbia Tel/Fax: +381-(0)21-68-98-600 Mail: info@mi-system.co.rs, URL: http://www.mi-system.co.rs

Yes. From the Administrator's Reference: Dynamic Server does not use direct or concurrent I/O for cooked files used for temporary dbspace chunks. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) IIUG Board of Directors (art@iiug.org) See you at the 2010 IIUG Informix Conference April 25-28, 2010 Overland Park (Kansas City), KS www.iiug.org/conf Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Thu, May 6, 2010 at 1:44 PM, Ivan Zavis DBA <ivan.zavis@mi-system.co.rs>wrote: > Aha, I figured out (I think) ... > > DIRECT_IO parameter in onconfig says: Enables (1) or disables (0) direct > I/O for chunks > is using with ONLY for cooked files and only if that cooked file is NOT > using for tempdbs .... > > I'm I right ? > > Thanks, > > Ivan. > > On 06.05.2010 19:09, Art Kagel wrote: > > First, you are not using DIRECT_IO with the RAW device, it is not > necessary > > as RAW doesn't go through the linux buffer cache anyway. In addition, > note > > that you are not using KAIO for the tempdbs2 file so it is not using > > DIRECT_IO either. That means that, yes, you are seeing the effect of > > rereading the Linux buffer cache over and over. IDS writes to filesystem > > chunks using either DIRECT_IO or O_SYNC so that the buffer linux cache > pages > > are either bypassed or flushed synchronously. Writes under DIRECT_IO are > > about 5% slower than to RAW in general and under O_SYNC they are as much > as > > 25% slower than RAW in general. However, for temp tables that are written > > once and reread many times, there can be benefits. > > > > Art > > > > Art S. Kagel > > Advanced DataTools (www.advancedatatools.com) > > IIUG Board of Directors (art@iiug.org) > > > > See you at the 2010 IIUG Informix Conference > > April 25-28, 2010 > > Overland Park (Kansas City), KS > > www.iiug.org/conf > > > > Disclaimer: Please keep in mind that my own opinions are my own opinions > and > > do not reflect on my employer, Advanced DataTools, the IIUG, nor any > other > > organization with which I am associated either explicitly, implicitly, or > by > > inference. Neither do those opinions reflect those of other individuals > > affiliated with any entity with which I am affiliated nor those of the > > entities themselves. > > > > On Thu, May 6, 2010 at 11:51 AM, IVAN > ZAVIS<ivan.zavis@mi-system.co.rs>wrote: > > > > > >> I have two temp dbspaces on same disk. One is using like raw devices > with > >> direct I/O and another like file in Linux file system. Both of them have > >> almost same number of read/write but, using onstat -g iof command for > >> monitoring system, looks like temp dbspace in file system is 100 times > >> faster > >> than raw dbspace ? > >> > >> Is this bug or this is because of Linux cache ? > >> > >> If it tempdbs on file system is really 100 times faster (when is talking > >> about > >> of small amount of data to read and write, but many many times), then it > is > >> good to use temp dbspace in file system instead in raw devices . > >> > >> Here it is, what onstat -g iof says: > >> > >> IBM Informix Dynamic Server Version 11.50.FC4WE ... > >> > >> AIO global files: > >> gfd pathname bytes read page reads bytes write page writes io/s > >> .... > >> 5 /dev/raw3 6542811136 3194732 6865305600 3352204 125.1 > >> > >> op type count avg. time > >> > >> seeks 0 N/A > >> > >> reads 0 N/A > >> > >> writes 109 0.0005 > >> > >> kaio_reads 119218 0.0034 > >> > >> kaio_writes 141296 0.0119 > >> > >> 6 tempdbs2 6508036096 3177752 6859382784 3349308 15970.2 > >> > >> op type count avg. time > >> > >> seeks 0 N/A > >> > >> reads 117239 0.0000 > >> > >> writes 137252 0.0001 > >> > >> kaio_reads 0 N/A > >> > >> kaio_writes 0 N/A > >> > >> > >> > >> > >> > > > > ******************************************************************************* > > > >> Forum Note: Use "Reply" to post a response in the discussion forum. > >> > >> > >> > > --001636e0a66c16cce50485f0029c > > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > Ivan Zaviç > System& DB Administrator > Mobile:+381-64-846-99-08 > mail: ivan.zavis@mi-system.co.rs > _________________________________________________________ > > M&I SYSTEMS CO. > Cirila i Metodija 13/a, 21000 Novi Sad, Serbia > Tel/Fax: +381-(0)21-68-98-600 > Mail: info@mi-system.co.rs, URL: http://www.mi-system.co.rs > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --001636e0a649abe6710485f0e0f3