Re: Raw vs. Cooked files

HOWEVER, Informix IDS does not user the OS buffering. It opens all COOKED chunks with O_SYNC which bypasses most of the benefits of the buffering in favor of added data integrity. Art S. Kagel ----- Original Message ----- From: Martin Fuerderer <MARTINFU@de.ibm.com> At: 5/18 4:27 Hi, I'd say the correct answer to the question is: "It depends." -------------------------------------------------------------------- The implementation of file system I/O buffering by the OS can be very efficient on Linux. But for that the OS needs sufficient memory for the buffering. And this is what makes the "It depends". Therefore there are 2 extremes: a) almost all physical memory is used by IDS in the form of shared memory (allocated SHM segments listed by "onstat -g seg"). A good indication for this scenario is a machine that uses little swap space, because it already has swapped out some minor "applications". The system is at the (sensible) limit of its memory usage. In this scenario, raw devices and the utilization of KAIO (available with IDS 10.00.UC1 on Linux) will in general perform better than cooked files. b) there's a lot of unused physical memory left over, even though IDS is configured optimally (e.g. BUFFERS is big enough, etc.). A recent example of such a scenario was 32-bit IDS installed on a 64-bit Linux, where IDS can use only ~4 GB of the available 24 GB of physical memory. In such a scenario, the OS can allocate a lot of memory for file system I/O buffering and this will make file system I/O (i.e. for chunks on cooked files) very efficient. In this scenario it will normally perform better than raw devices and KAIO. This is the general experience on Linux, where cooked file chunks are on a non-journalling file system. Not all journalling file systems are un-suitable for IDS chunks, but "reiserfs" certainly is un-suitable, so make sure you stay clear of that. E.g. use the "ext2" file system instead. Unix flavours other than Linux may exhibit completely different behaviour regarding raw devices vs. cooked files. This is mainly because they use different implementations of file systems and their I/O buffering. These things cannot be compared across different operating systems. -------------------------------------------------------------------- I think I've written an e-mail to this effect before to this list, but I'm not sure. So whoever feels responsible for some certain FAQ pages, please feel free to cut&paste the above text between the "---------" separators into the FAQ (with or without citing my name ...). But make sure that the text is complete ! Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management owner-informix-list@iiug.org wrote on 18.05.2005 00:40:08: > Rajesh Kapur wrote: > > Hello, > > Chec out last weeks postings here where this subject was beaten to death. I > say YES! Others say NO! Bottom line? Test it yourself. The article you > quote in the FAQ tells you how I tested it. > > Art S. Kagel > > > I am installing IDS 10 on RHEL3 on an intel x86 server. I need to make the > > choice between raw devices and cooked files. > > I found the following information in the FAQ's. Is this information still > > accurate? Do cooked files continue to be significantly slower than the raw > > files? I have heard that UNIX file systems have improved over the years and > > the performance divide between raw and cooked files has narrowed > > considerabley. Is it true? > > > > Thanks! > > - Rajesh > > -------------------------------------------------------------------------------- ------ > > > > 6.38 Is raw disk faster than cooked files? > > On 22nd Jun 1998 kagel@bloomberg.net (Art S. Kagel) wrote:- > > > > ....................the safety issue of cooked files is no longer a problem. > > The big problem with cooked files still is performance. All writes and reads > > to/from cooked files MUST go through the UNIX buffer cache. This means an > > additional copy from the server's output buffer to the UNIX buffer page then > > a synchronous write to disk. This is opposed to a write to a way file where > > the server's output buffer is written directly to disk without the > > intervening copy. This is just faster. Anyone who has written anything that > > can test this can attest to the difference in speed. Here are my test > > results: > > > > > > FileType Sync? Times (real/user/system) 2run avg > > --------------- ----- ----------------------------------------- > > Filesystem file N 14.40/3.70/2.52 > > Y 15.02/3.61/2.63 > > Cooked disk N 12.81/3.74/2.24 > > Y 13.42/3.84/2.43 > > Raw disk N 9.32/3.67/1.52 > > Y 9.40/3.66/1.44 > > > > From this you can clearly see the cost of Cooked files and of synced cooked > > files. The tests were done with a version of my ul.ec utility modified to > > optionally open the output file O_SYNC. Cooked disk partition is almost 50% > > slower than raw disk partition and cooked filesystem files are almost 60% > > slower. The penalty for O_SYNC is an additional 5% for cooked files and > > negligible for RAW files (as expected). The test file was 2.85MB written > > using 4K I/O pages (the default fopen application buffer size) which should > > simulate Informix performance. The Cooked and Raw disk partition tests were > > conducted to a singleton 9GB Fast Wide SCSI II drive using the raw and > > cooked device files corresponding to the same drive. > > > > sending to informix-list