Raw device with Suse Linux Enterpise 10 and INform
Sven asked whether IDS 10 on SLES 10 automatically uses O_DIRECT on LVM2 partitions, since the Linux raw device module was deprecated in kernel 2.6.16. Answers: IDS 10 only uses O_DIRECT with block (not cooked) devices when KAIO is on; the DIRECT_IO onconfig parameter (not for temp dbspaces) arrived with IDS 11.10. It was noted that Linus had since un-deprecated the raw driver. Advice: prefer raw devices for production (referenced via symbolic links so you can switch to block devices plus direct I/O later), with Kagel ranking raw > O_DIRECT on cooked devices > O_DIRECT filesystems > cooked; ext3 with data=ordered plus direct I/O is fine for development.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Hi, the raw device module is depreceated with Kernel 2.6.16. Everyone is talking about O_DIRECT access. Question: IDS10 is setting O_DIRECT automatically when I am using an LVM2 Partition? Thanks Sven ---------------------------------------------- Sven Schuran - Maske AG - IT Operations Telefon: +49 (0) 40 / 88 166 - 283 Telefax: +49 (0) 40 / 88 166 - 5283 E-Mail : sschuran@maske.de <mailto:sschuran@maske.de>
Schuran, Sven wrote: > Hi, > > the raw device module is depreceated with Kernel 2.6.16. > Everyone is talking about O_DIRECT access. > > Question: IDS10 is setting O_DIRECT automatically when I am using an LVM2 > Partition? > Direct IO support is off by default. You have to set "DIRECT_IO 1" in your ONCONFIG file to enable it. However, I'm not certain it's even available in IDS 10.00, IB that the first release to support direct_io is IDS 11.10xC1. Anyway also note, from the Administrator's Guide for IDS v 11.10: > DIRECT_IO is not used for temporary dbspaces and can only be used for > dbspace chunks whose file systems support direct I/O for the page size. Art S. Kagel Oninit > Thanks Sven > > ---------------------------------------------- > Sven Schuran - Maske AG - IT Operations > > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
Art, Sven, IDS 10 uses O_DIRECT with block devices (not cooked files!!) when KAIO is ON. IDS 11 can use O_DIRECT for both cooked files and block devices, and the use of DIRECT IO is controlled by the $ONCONFIG parameter. From my own experience, RAW devices on Linux much work better with IDS 11, then DIRECT IO with block devices: from OS diagnostics I can clearly see, that in some situations (most probably, Informix disk sorts in temp dbspace), Linux buffer cache size abruptly increases. -Alexey Kagel (Oninit LLC) Schuran, Sven wrote: > Hi, > > the raw device module is depreceated with Kernel 2.6.16. > Everyone is talking about O_DIRECT access. > > Question: IDS10 is setting O_DIRECT automatically when I am using an LVM2 > Partition? > Direct IO support is off by default. You have to set "DIRECT_IO 1" in your ONCONFIG file to enable it. However, I'm not certain it's even available in IDS 10.00, IB that the first release to support direct_io is IDS 11.10xC1. Anyway also note, from the Administrator's Guide for IDS v 11.10: > DIRECT_IO is not used for temporary dbspaces and can only be used for > dbspace chunks whose file systems support direct I/O for the page size. Art S. Kagel Oninit > Thanks Sven > > ---------------------------------------------- > Sven Schuran - Maske AG - IT Operations > >
Alexey Sonkin wrote: > Art, Sven, > > IDS 10 uses O_DIRECT with block devices (not cooked files!!) > when KAIO is ON. > > IDS 11 can use O_DIRECT for both cooked files and block devices, > and the use of DIRECT IO is controlled by the $ONCONFIG parameter. > > >From my own experience, RAW devices on Linux much work better with IDS > 11, > then DIRECT IO with block devices: from OS diagnostics I can clearly > see, > that in some situations (most probably, Informix disk sorts in temp > dbspace), Linux buffer cache size abruptly increases. > Great! I always suspected that RAW is still better. The question is how much better and is it worth the effort. Now if we can just convince Linus to not drop RAW disk support in Linux based on that ... Art S. Kagel Oninit > -Alexey > > Kagel (Oninit LLC) > > Schuran, Sven wrote: > >> Hi, >> >> the raw device module is depreceated with Kernel 2.6.16. >> Everyone is talking about O_DIRECT access. >> >> Question: IDS10 is setting O_DIRECT automatically when I am using an >> > LVM2 > >> Partition? >> >> > > Direct IO support is off by default. You have to set "DIRECT_IO 1" in > your ONCONFIG file to enable it. > > However, I'm not certain it's even available in IDS 10.00, IB that the > first release to support direct_io is IDS 11.10xC1. Anyway also note, > from the Administrator's Guide for IDS v 11.10: > > >> DIRECT_IO is not used for temporary dbspaces and can only be used for >> dbspace chunks whose file systems support direct I/O for the page >> > size. > > Art S. Kagel > Oninit > >> Thanks Sven >> >> ---------------------------------------------- >> Sven Schuran - Maske AG - IT Operations >> > > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
Hi, I am setting up a new system. SLES10 with IDS10, what should I prefer raw devices with the raw module or O_DIRECT support. Sven -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Art S. Kagel (Oninit LLC) Sent: Wednesday, January 16, 2008 9:56 PM To: ids@iiug.org Subject: Re: Raw device with Suse Linux Enterpise 10 an.... [10958] Alexey Sonkin wrote: > Art, Sven, > > IDS 10 uses O_DIRECT with block devices (not cooked files!!) > when KAIO is ON. > > IDS 11 can use O_DIRECT for both cooked files and block devices, > and the use of DIRECT IO is controlled by the $ONCONFIG parameter. > > >From my own experience, RAW devices on Linux much work better with IDS > 11, > then DIRECT IO with block devices: from OS diagnostics I can clearly > see, > that in some situations (most probably, Informix disk sorts in temp > dbspace), Linux buffer cache size abruptly increases. > Great! I always suspected that RAW is still better. The question is how much better and is it worth the effort. Now if we can just convince Linus to not drop RAW disk support in Linux based on that ... Art S. Kagel Oninit > -Alexey > > Kagel (Oninit LLC) > > Schuran, Sven wrote: > >> Hi, >> >> the raw device module is depreceated with Kernel 2.6.16. >> Everyone is talking about O_DIRECT access. >> >> Question: IDS10 is setting O_DIRECT automatically when I am using an >> > LVM2 > >> Partition? >> >> > > Direct IO support is off by default. You have to set "DIRECT_IO 1" in > your ONCONFIG file to enable it. > > However, I'm not certain it's even available in IDS 10.00, IB that the > first release to support direct_io is IDS 11.10xC1. Anyway also note, > from the Administrator's Guide for IDS v 11.10: > > >> DIRECT_IO is not used for temporary dbspaces and can only be used for >> dbspace chunks whose file systems support direct I/O for the page >> > size. > > Art S. Kagel > Oninit > >> Thanks Sven >> >> ---------------------------------------------- >> Sven Schuran - Maske AG - IT Operations >> > > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ =========== ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum. See you at the IIUG Informix 2008 Conference The Power Conference for Informix Professionals April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas http://www.iiug.org/conf Registration Now Open!! -------------------------------------------------------------------- Sie finden uns im Internet unter: www.maske.de Maske AG Behringstraße 120 22763 Hamburg Handelsregister Hamburg HRB 83369 USt-Ident.-Nr. DE 116 923 883 Vorstand: Andreas Maske Vors. d. Aufsichtsrates: Wolfgang Hönen Diese e-mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese e-mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene e-mail. Vielen Dank. This e-mail may contain trade secrets or privileged, undisclosed or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.
Hi, > From my own experience, RAW devices on Linux much work better with IDS > 11, > then DIRECT IO with block devices: from OS diagnostics I can clearly > see, > that in some situations (most probably, Informix disk sorts in temp > dbspace), Linux buffer cache size abruptly increases. this is not a linux problem, this seems to be an Informix problem, cause of: >> DIRECT_IO is not used for temporary dbspaces and can only be used for >> dbspace chunks whose file systems support direct I/O for the page Sven -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Art S. Kagel (Oninit LLC) Sent: Wednesday, January 16, 2008 9:56 PM To: ids@iiug.org Subject: Re: Raw device with Suse Linux Enterpise 10 an.... [10958] Alexey Sonkin wrote: > Art, Sven, > > IDS 10 uses O_DIRECT with block devices (not cooked files!!) > when KAIO is ON. > > IDS 11 can use O_DIRECT for both cooked files and block devices, > and the use of DIRECT IO is controlled by the $ONCONFIG parameter. > > >From my own experience, RAW devices on Linux much work better with IDS > 11, > then DIRECT IO with block devices: from OS diagnostics I can clearly > see, > that in some situations (most probably, Informix disk sorts in temp > dbspace), Linux buffer cache size abruptly increases. > Great! I always suspected that RAW is still better. The question is how much better and is it worth the effort. Now if we can just convince Linus to not drop RAW disk support in Linux based on that ... Art S. Kagel Oninit > -Alexey > > Kagel (Oninit LLC) > > Schuran, Sven wrote: > >> Hi, >> >> the raw device module is depreceated with Kernel 2.6.16. >> Everyone is talking about O_DIRECT access. >> >> Question: IDS10 is setting O_DIRECT automatically when I am using an >> > LVM2 > >> Partition? >> >> > > Direct IO support is off by default. You have to set "DIRECT_IO 1" in > your ONCONFIG file to enable it. > > However, I'm not certain it's even available in IDS 10.00, IB that the > first release to support direct_io is IDS 11.10xC1. Anyway also note, > from the Administrator's Guide for IDS v 11.10: > > >> DIRECT_IO is not used for temporary dbspaces and can only be used for >> dbspace chunks whose file systems support direct I/O for the page >> > size. > > Art S. Kagel > Oninit > >> Thanks Sven >> >> ---------------------------------------------- >> Sven Schuran - Maske AG - IT Operations >> > > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ =========== ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum. See you at the IIUG Informix 2008 Conference The Power Conference for Informix Professionals April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas http://www.iiug.org/conf Registration Now Open!! -------------------------------------------------------------------- Sie finden uns im Internet unter: www.maske.de Maske AG Behringstraße 120 22763 Hamburg Handelsregister Hamburg HRB 83369 USt-Ident.-Nr. DE 116 923 883 Vorstand: Andreas Maske Vors. d. Aufsichtsrates: Wolfgang Hönen Diese e-mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese e-mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene e-mail. Vielen Dank. This e-mail may contain trade secrets or privileged, undisclosed or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.
On Wednesday 16 January 2008 21:56, Art S. Kagel (Oninit LLC) wrote: > Alexey Sonkin wrote: > > Art, Sven, > > > > IDS 10 uses O_DIRECT with block devices (not cooked files!!) > > when KAIO is ON. > > > > IDS 11 can use O_DIRECT for both cooked files and block devices, > > and the use of DIRECT IO is controlled by the $ONCONFIG parameter. > > > > >From my own experience, RAW devices on Linux much work better with > > > IDS > > > > 11, > > then DIRECT IO with block devices: from OS diagnostics I can > > clearly see, > > that in some situations (most probably, Informix disk sorts in temp > > dbspace), Linux buffer cache size abruptly increases. > > Great! I always suspected that RAW is still better. The question is > how much better and is it worth the effort. Now if we can just > convince Linus to not drop RAW disk support in Linux based on that FYI, Linus already removed the deprecated status of the raw driver, see http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=abd4 aa5a97ebc0efb9a7fbc98ef0bcf39266fadf Andreas > ... > > Art S. Kagel > Oninit > > > -Alexey > > > > Kagel (Oninit LLC) > > > > Schuran, Sven wrote: > >> Hi, > >> > >> the raw device module is depreceated with Kernel 2.6.16. > >> Everyone is talking about O_DIRECT access. > >> > >> Question: IDS10 is setting O_DIRECT automatically when I am using > >> an > > > > LVM2 > > > >> Partition? > > > > Direct IO support is off by default. You have to set "DIRECT_IO 1" > > in your ONCONFIG file to enable it. > > > > However, I'm not certain it's even available in IDS 10.00, IB that > > the first release to support direct_io is IDS 11.10xC1. Anyway also > > note, > > > > from the Administrator's Guide for IDS v 11.10: > >> DIRECT_IO is not used for temporary dbspaces and can only be used > >> for dbspace chunks whose file systems support direct I/O for the > >> page > > > > size. > > > > Art S. Kagel > > Oninit > > > >> Thanks Sven > >> > >> ---------------------------------------------- > >> Sven Schuran - Maske AG - IT Operations > > ===================================================================== >====================== Please access the attached hyperlink for an > important electronic communications disclaimer: > > http://www.oninit.com/home/disclaimer.php > > > ===================================================================== >====================== > > > ********************************************************************* >********** Forum Note: Use "Reply" to post a response in the > discussion forum. > > See you at the IIUG Informix 2008 Conference > The Power Conference for Informix Professionals > April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas > http://www.iiug.org/conf > Registration Now Open!! -- Andreas Breitfeld; Informix Development Munich IBM Deutschland GmbH; Vorsitzender des Aufsichtsrats: Hans Ulrich Maerki; Geschäftsführung: Martin Jetter (Vorsitzender), Rudolf Bauer, Christian Diedrich, Christoph Grandpierre, Matthias Hartmann, Thomas Fell, Michael Diemer; Sitz der Gesellschaft: Stuttgart; Registergericht: Amtsgericht Stuttgart, HRB 14562 WEEE-Reg.-Nr. DE 99369940
Schuran, Sven wrote: > Hi, > > I am setting up a new system. > > SLES10 with IDS10, what should I prefer raw devices with the raw module > or O_DIRECT support. > > Sven, Personally I still prefer RAW device. It's more secure and even a bit or more faster than O_DIRECT. I have softened my position on RAW somewhat as the benefits of RAW over O_DIRECT are smaller than over normal COOKED devices or filesystems and because of the rumor that Linus had again deprecated RAW devices. But, given that RAW is alive on Linux again, I'll go back to basics. Here's the chunk placement hierarchy according to Kagel: RAW device files are best O_DIRECT on COOKED device files is next. O_DIRECT on filesystems that support it is third COOKED device files is fourth COOKED filesystem files on non-journaled filesystems are dead last Jounaled filesystems are to be avoided at all costs Note that while I have extensively tested RAW -vs- COOKED -vs- filesystems I have not myself tested O_DIRECT I/O and am relying on the reports of others -- for now. Art S. Kagel Oninit > Sven > > <SNIP> > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
Andreas Breitfeld wrote: > On Wednesday 16 January 2008 21:56, Art S. Kagel (Oninit LLC) wrote: > >> Alexey Sonkin wrote: >> >>> Art, Sven, >>> >>> IDS 10 uses O_DIRECT with block devices (not cooked files!!) >>> when KAIO is ON. >>> >>> IDS 11 can use O_DIRECT for both cooked files and block devices, >>> and the use of DIRECT IO is controlled by the $ONCONFIG parameter. >>> >>> >From my own experience, RAW devices on Linux much work better with >>> >>>> IDS >>>> >>> 11, >>> then DIRECT IO with block devices: from OS diagnostics I can >>> clearly see, >>> that in some situations (most probably, Informix disk sorts in temp >>> dbspace), Linux buffer cache size abruptly increases. >>> >> Great! I always suspected that RAW is still better. The question is >> how much better and is it worth the effort. Now if we can just >> convince Linus to not drop RAW disk support in Linux based on that >> > FYI, Linus already removed the deprecated status of the raw driver, see > > http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=abd4 aa5a97ebc0efb9a7fbc98ef0bcf39266fadf > > Very cool. I'd missed that announcement. Art S. Kagel Oninit > Andreas > > >> ... >> >> Art S. Kagel >> Oninit >> >> >>> -Alexey >>> >>> Kagel (Oninit LLC) >>> >>> Schuran, Sven wrote: >>> >>>> Hi, >>>> >>>> the raw device module is depreceated with Kernel 2.6.16. >>>> Everyone is talking about O_DIRECT access. >>>> >>>> Question: IDS10 is setting O_DIRECT automatically when I am using >>>> an >>>> >>> LVM2 >>> >>> >>>> Partition? >>>> >>> Direct IO support is off by default. You have to set "DIRECT_IO 1" >>> in your ONCONFIG file to enable it. >>> >>> However, I'm not certain it's even available in IDS 10.00, IB that >>> the first release to support direct_io is IDS 11.10xC1. Anyway also >>> note, >>> >>> from the Administrator's Guide for IDS v 11.10: >>> >>>> DIRECT_IO is not used for temporary dbspaces and can only be used >>>> for dbspace chunks whose file systems support direct I/O for the >>>> page >>>> >>> size. >>> >>> Art S. Kagel >>> Oninit >>> >>> >>>> Thanks Sven >>>> >>>> ---------------------------------------------- >>>> Sven Schuran - Maske AG - IT Operations >>>> >> ========================= > ================================================================================ =========== Please access the attached hyperlink for an important electronic communications disclaimer: http://www.oninit.com/home/disclaimer.php ================================================================================ ===========
Art, "journaled" file systems should be split into two categories: 1. File systems with full journaling (e.g. JFS (aka VxFS) or EXT3, when mounted with ' data=journal' flag) 2. File systems with metadata-only journaling (e.g. EXT3 mounted with 'data=ordered') In the second case, the journaling only slows down the creation of a file, and doesn't effect the 'normal' Informix operation. On the other hand, journaling gives you some extra FS protection, and improves 'fsck' speed after non-clean reboot... I personally prefer EXT3 with 'data=ordered' to EXT2. Also, without Direct IO, Cooked (aka block) devices have bigger overhead, then cooked file system files (might sound strange..) The reason is that OS cache block size is smaller for block devices, then for cooked files: on Suse 9, 4k for files and 512 bytes for block devices. Each OS cache block has an overhead of 96 bytes - size of block header structure. Clear enough, that, for smaller blocks, the overhead is more significant So, without Direct IO, I'd prefer 'cooked file system files' to 'cooked (block) devices' To summarize the discussion, I'd suggest the following: For production system, use RAW devices with Informix, but reference them through symbolic links. If, in the future, RAW devices become deprecated, switch the links to corresponding block devices, and continue with Direct IO. For development systems, one can go with EXT3 filesystem, mounted with 'data=ordered', and Direct IO. -Alexey > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Art S. > Kagel (Oninit LLC) > > Schuran, Sven wrote: > > Hi, > > > > I am setting up a new system. > > > > SLES10 with IDS10, what should I prefer raw devices with the raw module > > or O_DIRECT support. > > > > > > Sven, > > Personally I still prefer RAW device. It's more secure and even a bit > or more faster than O_DIRECT. I have softened my position on RAW > somewhat as the benefits of RAW over O_DIRECT are smaller than over > normal COOKED devices or filesystems and because of the rumor that Linus > had again deprecated RAW devices. But, given that RAW is alive on Linux > again, I'll go back to basics. Here's the chunk placement hierarchy > according to Kagel: > > RAW device files are best > O_DIRECT on COOKED device files is next. > O_DIRECT on filesystems that support it is third > COOKED device files is fourth > COOKED filesystem files on non-journaled filesystems are dead last > Jounaled filesystems are to be avoided at all costs > > Note that while I have extensively tested RAW -vs- COOKED -vs- > filesystems I have not myself tested O_DIRECT I/O and am relying on the > reports of others -- for now. > > Art S. Kagel > Oninit > > > Sven > > > > <SNIP> > > > > > ======================================================================== ====== > ============= > Please access the attached hyperlink for an important electronic > communications disclaimer: > > http://www.oninit.com/home/disclaimer.php > > > ======================================================================== ====== > ============= > > > ************************************************************************ ****** > * > Forum Note: Use "Reply" to post a response in the discussion forum. > > See you at the IIUG Informix 2008 Conference > The Power Conference for Informix Professionals > April 27 - 30, 2008 Marriott Overland Park (Kansas City), Kansas > http://www.iiug.org/conf > Registration Now Open!!