Cooked files vs Raw disc

I has a bit of a discussion with one of our developers today and the topic of cooked vs raw file systems came up, and I am looking for opinions. Reason 1 for not using raw disc was that a 10% to 15% reduction in performance can be made up in hardware using a caching controller with lots of memory. Also the time involved in setting up and maintaining raw partitions is to time consuming. Reason 2 for not using raw disc was that if you use cooked files when its time to move from the test environment to production you can just TAR or ZIP up the cooked files and transfer them to production and you are guaranteed that the tested TEST environment get to production. Thoughts? Thanks ---------- CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,is for the sole use of the intended recipient(s), even if addressed incorrectly, and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy or delete all copies of the original message and all attachments, including deletion from the trash or equivalent folder. Thank you.

Peter, Our experience was that Informix on raw devices was about 10 times faster, not 10% (this was on Solaris servers). Setting up the raw partitions was no more demanding of my time than was doing cooked devices (although our sysad had to do some work to set up the raw devices). The speed difference was sufficient to compensate for any other difficulties. Our backups were usually either through informix utilities or by specific scripts, so I can't really address the virtues of your second point -- just remember to bring down Informix completely before making the copy of such cooked files. At least historically, raw devices were considered to be slightly safer than cooked as Informix does not have to go through an OS call with whatever uncertainties that has about the data actually being committed to disk -- with raw devices there's no such doubt since Informix is in direct control. Again, this may different in other operating systems. HTH, Greg Williamson DBA GlobeXplorer LLC -----Original Message----- From: Peter J Dia.... [mailto:pdiazdeleon@infinityhealthcare.com] Sent: Thu 12/9/2004 11:28 AM To: ids@iiug.org Cc: Subject: Cooked files vs Raw disc [3863] I has a bit of a discussion with one of our developers today and the topic of cooked vs raw file systems came up, and I am looking for opinions. Reason 1 for not using raw disc was that a 10% to 15% reduction in performance can be made up in hardware using a caching controller with lots of memory. Also the time involved in setting up and maintaining raw partitions is to time consuming. Reason 2 for not using raw disc was that if you use cooked files when its time to move from the test environment to production you can just TAR or ZIP up the cooked files and transfer them to production and you are guaranteed that the tested TEST environment get to production. Thoughts? Thanks ---------- CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,is for the sole use of the intended recipient(s), even if addressed incorrectly, and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy or delete all copies of the original message and all attachments, including deletion from the trash or equivalent folder. Thank you.

I agree with you. I Have 15 years of experience with informix & UNIX. Another thing is that if you user cooked files and the system crashes, in the startup cooked files could get lost because the fsck cleaning ! And sure you must bring your system down if you try to do backups with tar or cpio (otherwise will be inconsistent backup ! ) -----Mensaje original----- De: forum.subscriber@iiug.org [mailto:forum.subscriber@iiug.org] En nombre de Gregory S. .... Enviado el: Jueves, 09 de Diciembre de 2004 03:10 p.m. Para: ids@iiug.org Asunto: RE: Cooked files vs Raw disc [3864] Peter, Our experience was that Informix on raw devices was about 10 times faster, not 10% (this was on Solaris servers). Setting up the raw partitions was no more demanding of my time than was doing cooked devices (although our sysad had to do some work to set up the raw devices). The speed difference was sufficient to compensate for any other difficulties. Our backups were usually either through informix utilities or by specific scripts, so I can't really address the virtues of your second point -- just remember to bring down Informix completely before making the copy of such cooked files. At least historically, raw devices were considered to be slightly safer than cooked as Informix does not have to go through an OS call with whatever uncertainties that has about the data actually being committed to disk -- with raw devices there's no such doubt since Informix is in direct control. Again, this may different in other operating systems. HTH, Greg Williamson DBA GlobeXplorer LLC -----Original Message----- From: Peter J Dia.... [mailto:pdiazdeleon@infinityhealthcare.com] Sent: Thu 12/9/2004 11:28 AM To: ids@iiug.org Cc: Subject: Cooked files vs Raw disc [3863] I has a bit of a discussion with one of our developers today and the topic of cooked vs raw file systems came up, and I am looking for opinions. Reason 1 for not using raw disc was that a 10% to 15% reduction in performance can be made up in hardware using a caching controller with lots of memory. Also the time involved in setting up and maintaining raw partitions is to time consuming. Reason 2 for not using raw disc was that if you use cooked files when its time to move from the test environment to production you can just TAR or ZIP up the cooked files and transfer them to production and you are guaranteed that the tested TEST environment get to production. Thoughts? Thanks ---------- CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,is for the sole use of the intended recipient(s), even if addressed incorrectly, and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy or delete all copies of the original message and all attachments, including deletion from the trash or equivalent folder. Thank you.

O.K... I just can't stand it... First, you can NEVER "make up" the 10 to 15% difference; think about it. Yes, you can make things faster with a caching controller with lots of memory, but they'll always be 10 to 15% faster on the same hardware if you use raw space than if you use cooked space. To say that it is "time consuming" to set up raw partitions (unless you're on a Windows platform, and then I can't speak to raw space) demonstrates a lack of understanding in what is involved in setting up file systems. On any of the Unix platforms I've ever worked on, the work to set up a file system is slightly greater than setting up the raw logical volume (since file systems typically sit on top of raw volumes, this makes sense). So unless you're willing to be COMPLETELY ignorant about where your data resides, letting your system administrator just drop your database on whatever disk he/she happens to have handily lying about, you're not really saving yourself any effort. And then, you will have reduced the price/performance ratio of your hardware by the 10 to 15%. Second... I can't say that I've ever tried moving an instance using the method specified. I can't say that I think it's a good idea either. There are just too many ways that I think that could get hosed up. That said, there's no reason that you couldn't do this with raw space if you wanted to; the command is just a little different. With the cooked files, you'll have to do an FTP of the files from one system to the other; with raw space, you'd first do a dd if=<your_raw_device> of=<some_file_name> Then FTP the <some_file_name> file to the remote system, finally doing a dd if=<some_file_name> of=<remote_raw_device> and you've accomplished the same thing. I'll admit, it's a little more complex, but not all that much more complex. Further, by keeping your data in raw files, you are less likely to run into folks who think that you don't need to do database backups because, "We do an O/S backup every night, and those files are included in the backup"... or the system administrator who calls you saying, "Well, the /var/<your_database> file system was filling up, so I gzip'ed all those files there... I've gotten a few calls, and I was wondering if you could help me... " In short, raw files provide you with a "reminder" that you're dealing with things that should NOT be managed by the operating system in the first place. My experience has been that people are resistant to raw devices at first, because they are not accustomed to them, but once you've implemented them, folks begin to realize that there's very little difference in the administration of the database with raw vs cooked files, and you get that bonus 10 to 15% performance boost. Hard to argue against it... Dan Michaelis Database Administrator/Developer eOriginal 351 West Camden Street Suite 800 Baltimore, MD 21201 410.625.5187 (phone) 410.659.9799 (fax) -----Original Message----- From: Peter J Dia.... [mailto:pdiazdeleon@infinityhealthcare.com] Sent: Thursday, December 09, 2004 2:28 PM To: ids@iiug.org Subject: Cooked files vs Raw disc [3863] I has a bit of a discussion with one of our developers today and the topic of cooked vs raw file systems came up, and I am looking for opinions. Reason 1 for not using raw disc was that a 10% to 15% reduction in performance can be made up in hardware using a caching controller with lots of memory. Also the time involved in setting up and maintaining raw partitions is to time consuming. Reason 2 for not using raw disc was that if you use cooked files when its time to move from the test environment to production you can just TAR or ZIP up the cooked files and transfer them to production and you are guaranteed that the tested TEST environment get to production. Thoughts? Thanks ---------- CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,is for the sole use of the intended recipient(s), even if addressed incorrectly, and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy or delete all copies of the original message and all attachments, including deletion from the trash or equivalent folder. Thank you.

This is a question I have answered many times in the past 17 years. And the answer is always the same. If you use cooked files your performance will be slower as informix cannot take advantage of the contiguous disk access. And the 10 to 15% reduction in performance is assuming that you create "clean" cooked files. If the cooked file is on a well-used file system the performance overhead of cooked has been seen to be much more than 10 to 15%. But it isn't only that. When I look at any IDS system I look at issues like cooked versus raw, extent sizing, evidence of thought given to disk placement, no of logs, log sizing, config parameters, etc. If it is evident that "quick" answers have been used it is quite likely that the system hasn't been thought about very much at all. And if the system engineering hasn't been done then it is quite likely that the system is vulnerable to more problems. I am currently working with a number of systems that have not been engineered and they all seem to suffer from similar problems. Performance is very good - most of the time. And in one recent case the customer has upgraded their processor power by about 20 times, with faster disks, and more memory, and already within 3 weeks they are seeing performance oddities. And nobody did the system engineering because in testing it was like the proverbial "s**t off a shovel". I am now considering a shutdown to correct some little things that were overlooked. So, the advice of one of the longest serving Informix DBAs is - do the engineering. It might take longer but it's worht it in the long run. Regards Malcolm -----Original Message----- From: forum.subscriber@iiug.org [mailto:forum.subscriber@iiug.org] On Behalf Of Peter J Dia.... Sent: 09 December 2004 19:28 To: ids@iiug.org Subject: Cooked files vs Raw disc [3863] I has a bit of a discussion with one of our developers today and the topic of cooked vs raw file systems came up, and I am looking for opinions. Reason 1 for not using raw disc was that a 10% to 15% reduction in performance can be made up in hardware using a caching controller with lots of memory. Also the time involved in setting up and maintaining raw partitions is to time consuming. Reason 2 for not using raw disc was that if you use cooked files when its time to move from the test environment to production you can just TAR or ZIP up the cooked files and transfer them to production and you are guaranteed that the tested TEST environment get to production. Thoughts? Thanks ---------- CONFIDENTIALITY NOTICE: This e-mail message, including any attachments,is for the sole use of the intended recipient(s), even if addressed incorrectly, and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy or delete all copies of the original message and all attachments, including deletion from the trash or equivalent folder. Thank you.

malcolm wea.... said: > This is a question I have answered many times in the past 17 years. And > the answer is always the same. If you use cooked files your performance > will be slower as informix cannot take advantage of the contiguous disk > access. And the 10 to 15% reduction in performance is assuming that you > create "clean" cooked files. If the cooked file is on a well-used file > system the performance overhead of cooked has been seen to be much more > than 10 to 15%. No one has mentioned KAIO so far... it only works on raw disk. -- Bye now, Obnoxio "C'est pas parce qu'on n'a rien à dire qu'il faut fermer sa gueule" - Coluche "I'm trying to see things your way, but I can't get my head up my ass" - JCH "Ogni uomo mi guarda come se fossi una testa di cazzo" - Marco I went to the airport to check in and they asked what I did because I looked like a terrorist. I said I was a comedian. They said, "Say something funny then." I told them I had just graduated from flying school. -- Ahmed Ahmed