Re: Raw I/O vs files (was Re: Raw partitions / cooked files)

In article <2506@coyote.UUCP> gainer@slowmo.almaden.ibm.com (P. Gainer) writes: >In a recent append, lparsons@exlog.com (Lee Parsons) writes... >>I would argue that if you need the performance increase bad enough to >>start changing the filesystem code, you have no excuse not to skip it >>and go directly raw. > >Perhaps I was unclear. I usually am :) What I meant was that implementing >a storage architecture using raw I/O is a lot like implementing a >specialized file system. My questions are about the possibilities of getting >higher performance because of many things one might choose to do which >general purpose OS file systems don't do. For example: huge block size, >large sequential prefetch, shorter read/write pathlength etc. > No, I dont think you where unclear, I just have trouble abstracting myself from the problem enough to consider a situation where all of this work hasn't already been done. I have a bad habit of thinking of things in the what's available now mode. If we were to sit down and write a DBFS that incorporated all the things a database needed for good operation, how would that work? While I think this would make it easier for a firm to write a reasonable DBMS with out haveing to do the more intense file access code and that would be a good thing but, I don't see how that would translate to better performance. Yes, this situation would work better than a regular FS but it would still be suboptimal compared to no FS at all. Once again this is the trade off between ease of use and performance. Since we don't know what the perf gains are between FS vs DBFS vs raw, where do we go from here? I'm going to hold that for the most part the difference between FS and raw is nominal so the diff between DBFS and raw would be less than nominial. [...] >>I think that is because you, I and everybody else already know the >>answer. The only way your are going to see a *significantly* faster >>raw benchmark is if something else somewhere is not tuned well. > >I don't necessarily think this is true. If you look at the TPC-A/B/C >benchamrks, all the leaders get their numbers using raw I/O. This implies >raw I/O can be faster when tuned. Without actually doing a huge benchmark >comparison, I would be curious to know what others have observed. Hmmm. If I where to run a TCP benchmark I would set up a raw db also. Not because I knew that I would get any added performance but because all the reasons for not going raw are administrative, so for a short lived db why take the risk that the fscode might get in the way. I will admit though that in a full out I/O bound db, raw would have to be faster provided, that the db wasn't broken and did it's own cacheing/read ahead. But with a FS with large blocks, reasonable disk drives and controllers, I'm not clear where the BIG win is. [ once again I offer a contriversial and probably wrong opinion with out any numbers to support it. I should run for Senate. :-} ] In the cases where I have gone raw it has been because I have not been able to fix the things I though were really wrong. ie) what do you mean your running this database with only two disks! or Yes, your computer can have more than one disk controller in it. In this regard, I have been lucky that "acceptable" performance was obtainable via some other method. As such I have always seen going raw as the last step. -- Regards, Lee E. Parsons Baker Hughes Inteq, Inc Oracle Database Administrator lparsons@exlog.com