Re: The old raw devices chestnut.

Data Goob wrote: > I've found jfs to be "fast enough", considering we are moving > from SQL-Server, no raw-disks on that one ( chuckles and laughs > allowed :-) . The risks that raw-disks present makes me wonder > if they are worth it. Restoring raw-disk databases presents its > own set of problems, whereas regular files can be backed up and > restored more simply and with more flexibility. Ummmmm, if you try to archive regular files when the database is active, you *will* have an inconsistent snapshot of the database. Do you perform these backups only when the engine is offline? I can't see this being safe on any brand of engine if the database is "twinkling". The only way to get a consistent (ie logical instant in time) backup of a live, active database is to use a tool which maintains the illusion on your behalf.

Andrew, Fair enough considerations, and you hit the point on the head, the backup and restore. We have always set up backup of logs and data to a real backup, but considering all the options I might 'need' in the event of system failure, or migration, or cloning, raw devices are an added risk. We use a third-party backup tool to backup our SQL-Server databases, as well as EMC Timefinder to flash-copy databases into production environments. We will apply the same methods with DB2 or MySQL for that matter. The raw-disk paradigm adds only extra trouble/work. The real question is, how will you backup, much less restore raw device databases? Are you prepared to deal with the inflexibility it presents? Have you ever run Informix or other vendor database through a complete backup and restore, testing all the options with raw-device dbspaces? Most DBAs I've run into have never had to restore a database at any time in their career much less even test it. Is that amazing or what! Consider too, clustering of systems and disks, and the administrative challenges associated with that. Add raw devices to multitudes of servers and it becomes more risky and more to manage. It puts more opportunities for failure in the administration path, and nobody in their right mind wants to increase risk. Is the extra 10% in speed worth it? Maybe that 'speed' can come from somewhere else? One other thing you might find attractive to plain ole files instead of raw devices is the ability to clone databases. You can get quite creative with plain files in ways that you cannot with raw devices because of the lock-in of raw-devices. You create more work for yourself in the long run with raw disks, unless of course you're not as lazy as me and actually enjoy the extra work. '-) "Andrew Hamm" <ahamm@mail.com> wrote in message news:c5i1me$1vnic$1@ID-79573.news.uni-berlin.de... > Data Goob wrote: > > I've found jfs to be "fast enough", considering we are moving > > from SQL-Server, no raw-disks on that one ( chuckles and laughs > > allowed :-) . The risks that raw-disks present makes me wonder > > if they are worth it. Restoring raw-disk databases presents its > > own set of problems, whereas regular files can be backed up and > > restored more simply and with more flexibility. > > Ummmmm, if you try to archive regular files when the database is active, you > *will* have an inconsistent snapshot of the database. Do you perform these > backups only when the engine is offline? I can't see this being safe on any > brand of engine if the database is "twinkling". The only way to get a > consistent (ie logical instant in time) backup of a live, active database is > to use a tool which maintains the illusion on your behalf. > >

"Data Goob" <datagoob@hotmail.com> wrote > One other thing you might find attractive to plain ole files > instead of raw devices is the ability to clone databases. You > can get quite creative with plain files in ways that you cannot > with raw devices because of the lock-in of raw-devices. Please elaborate. If you use symbolic links for raw devices, then there is no lock-in of raw devices, or am I missing something.

rkusenet wrote: > "Data Goob" <datagoob@hotmail.com> wrote > >> One other thing you might find attractive to plain ole files >> instead of raw devices is the ability to clone databases. You >> can get quite creative with plain files in ways that you cannot >> with raw devices because of the lock-in of raw-devices. > > > Please elaborate. > > If you use symbolic links for raw devices, then there is no lock-in > of raw devices, or am I missing something. I don't think so. Even with cooked files I'd be using symlinks; you never know when you need to put a lump onto another file system. Speaking only Informixly here, I can clone a live engine with symlinks and a simple procedure, and only a temporary outage to bounce the parent engine, so once again I don't see any administrative dramas with raw spaces. I think I just need to shrug and move on; nobody seems to have concrete facts to backup the claim. [about to reply to Data Goobs original message on that one - there's something in there that hints at something.] Also different engines will have different issues, so we've got to keep a careful distance from any specific engine on this cross-posted thread. Further, talking about restores rather than raw devices is straying a bit too far from the thread.

Data Goob wrote: > Andrew, > > Fair enough considerations, and you hit the point on > the head, the backup and restore. > > We have always set up backup of logs and data to a real > backup, but considering all the options I might 'need' > in the event of system failure, or migration, or cloning, > raw devices are an added risk. We use a third-party backup > tool to backup our SQL-Server databases, as well as EMC > Timefinder to flash-copy databases into production environments. > We will apply the same methods with DB2 or MySQL for that > matter. The raw-disk paradigm adds only extra trouble/work. OK - so you are talking about SQL-Server and various issues related to a backup tool you use? If so, we have no argument; I know nothing of your tools, and am speaking from an Informix + UNIX point of view. Our beloved symlinks on UNIX are not available to NT servers, so suggestions on this cannot help. I'm also of the opinion (only from reading) that unbuffered NTFS files are equivalent in performance and reliability to raw spaces on NT: Even with Informix on NT (of which i have almost no experience) symlinks are not available, and further, the recommendations from Informix states that you can use normal files (O/S buffered and capable of going onto FAT), or normal files on NTFS (which will be used unbuffered) or a raw partition on NT. Further, the documentation says that on NT, the use of unbuffered NTFS files is of equal performance to NT raw spaces, therefore it's not worth using raw spaces on NT. But that's NT, something I don't play with. > The real question is, how will you backup, much less restore > raw device databases? Are you prepared to deal with the > inflexibility it presents? Have you ever run Informix or > other vendor database through a complete backup and restore, > testing all the options with raw-device dbspaces? Absolutely. Sometimes under great duress. The use of raw spaces has always made the restores faster too. Jonathan Leffler (c.d.i stalwart and Informix Insider) has disputed the claim someone made that raw can be 5-10 TIMES faster, and I have to agree on that. Except for two points [once again, Informix+UNIX specific]: 1) during initialisation of an instance, if you initialise on cooked files, the creation of the dbspaces and especially the physical and logical logs takes an insufferable length of time. With raw spaces, creation of an engine is quite a few times faster. That means a great deal in the middle of a day's work. 2) During a restore, raw spaces are a few times faster. and 3) during a mass load of data, raw spaces are a few times faster. So is the proper use of PDQ and artificially inflated allocations of shared memory for a few unstated reasons (informix, once again...) 4) There is no point 4. All of these points are significant when major sequential writing is taking place. BTW I must also qualify that this experience applies to machines without fancy storage managers. If you are using a machine with SCSI disks directly connected to the SCSI bus or a straight-forward SCSI raid controller, then you'll notice performance benefits from using raw with Informix (any other engines?) If it's got a big fat storage manager, then it's implementation will hide the benefits, in which case, you need to ask "I'm using Engine E on platform P with storage manager SM, so what's the best storage model to use?" > Most DBAs > I've run into have never had to restore a database at any > time in their career much less even test it. Is that amazing > or what! It's more tragic than anything else. By a strange coincidence I'm in a discussion on this subject in another forum, and we're swapping horror stories, such as a customer who backed up to a cleaner tape for 2 weeks, or a customer who needed a restore and discovered that their 3 year old tapes cannot be read on their 5 year old tape drive which hasn't seen a cleaner tape ever.... I've had to do restores on customer sites who could not or would not afford disk mirrors, so we really did rely on the tapes for the redundancy. I've seen power supplies blow up. All sorts of things can and do happen. > Consider too, clustering of systems and disks, and the > administrative challenges associated with that. Add raw > devices to multitudes of servers and it becomes more risky > and more to manage. It puts more opportunities for failure > in the administration path, and nobody in their right mind > wants to increase risk. Is the extra 10% in speed worth it? Well, as I said, I don't understand the alleged extra administration. Clearly it's an NT thing? Or an issue for people who don't use the magic of symlinks? Pass. I'll stop asking now. > One other thing you might find attractive to plain ole files > instead of raw devices is the ability to clone databases. You > can get quite creative with plain files in ways that you cannot > with raw devices because of the lock-in of raw-devices. You > create more work for yourself in the long run with raw disks, > unless of course you're not as lazy as me and actually enjoy > the extra work. '-) This is where the YMMV slogan comes into play. I would never setup an Informix,UNIX,SCSI machine with anything else but symlinks and raw spaces. If I ever setup a machine with a high performance storage box, I'll look into the most appropriate mechanism for that. I'd probably setup any brand of engine on UNIX with symlinks, unless experience or advice shows that it's pointless. As for administration of raw spaces, on UNIX it's trivial. Meaningless. Not a problem. Do what your engine, backup tool and storage manager works best with. Unless someone else chips in with some detailed advice about other brands, the original poster will only be lurnin' about Informix engines today. If the OP wants more advice about Informix, please we should stop cross-posting and get into more detail only on comp.databases.informix, and stop boring the other newsgroups. Goob, I think you hang around c.d.i quite a bit, so if you wish, please further my education about the pain of raw spaces on c.d.i. Perhaps some specific stories are needed so I can undertand your experience.

"Andrew Hamm" <ahamm@mail.com> wrote in message news:c5i8fl$22chf$1@ID-79573.news.uni-berlin.de... > OK - so you are talking about SQL-Server and various issues related to a > backup tool you use? Yes, standard backup tools. > If so, we have no argument; I know nothing of your > tools, and am speaking from an Informix + UNIX point of view. Our beloved > symlinks on UNIX are not available to NT servers, so suggestions on this > cannot help. I'm also of the opinion (only from reading) that unbuffered > NTFS files are equivalent in performance and reliability to raw spaces on > NT: > SQL-Server is a dish best served, well, cooked. In most SQL-Server situations, probably 95% or more, SQL-Server is stored in plain ole files in a directory. You can create FILEGROUPS akin to containers and dbspaces, but in the real world most SQL-Server people haven't a clue about how to use FILEGROUPS so they simply stuff everything in one big default filegroup called PRIMARY. It's the equivalent in Informix of leaving everything in the rootdbs and never bothering with it, letting it get larger and larger. SQL-Server databases can be detached, that is, taken off-line, moved, copied, etc. But most people don't ever bother with disk layout except in the larger shops. As you suggest there are no linked files, this concept is completely opaque to Windows people they just don't connect with it. Incidentally if you use more than 16 files to build filegroups you cannot use the GUI to reattach a database--very cool thing to learn in a down-server situation. You instead have to use a script like the one below with the syntax FOR ATTACH at the end. Really spiffy. Example 1. Mydatabase with a lot of FILEGROUPS : CREATE DATABASE [Mydatabase] ON PRIMARY ( NAME = 'MYDB_PRIMARY_00' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_PRIMARY_00.MDF' , SIZE = 2048 MB , FILEGROWTH = 0% ) , FILEGROUP DATA ( NAME = 'MYDB_01' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_01.NDF' , SIZE = 4172 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_02' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_02.NDF' , SIZE = 4483 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_03' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_03.NDF' , SIZE = 3887 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_04' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_04.NDF' , SIZE = 3991 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_05' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_05.NDF' , SIZE = 7964 MB , FILEGROWTH = 20% ) FILEGROUP IDX ( NAME = 'MYDB_IDX_01' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_IDX_01.NDF' , SIZE = 2048 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_IDX_02' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_IDX_02.NDF' , SIZE = 2048 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_IDX_03' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_IDX_03.NDF' , SIZE = 2048 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_IDX_04' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_IDX_04.NDF' , SIZE = 2048 MB , FILEGROWTH = 20% ) , ( NAME = 'MYDB_IDX_05' , FILENAME = 'R:\\DATA\\MYDB_Data\\MYDB_IDX_05.NDF' , SIZE = 2048 MB , FILEGROWTH = 20% ) LOG ON ( NAME = 'MYDB_LOG_01' , FILENAME = 'R:\\DATA\\MYDB_Logs\\MYDB_LOG_01.LDF' , SIZE = 56 MB , FILEGROWTH = 10% ) GO Example 2. Mydatabase with no thought or plan, out of the box: CREATE DATABASE [A] ON PRIMARY ( NAME = 'a_Data' , FILENAME = 'R:\\Data\\A\\DATA\\A_Primary.MDF' , SIZE = 4 MB , FILEGROWTH = 10% ) , LOG ON ( NAME = 'a_Log' , FILENAME = 'R:\\DATA\\A\\LOG\\A_Log.ldf' , SIZE = 14 MB , FILEGROWTH = 10% ) GO Big bummer on FILEGROUPS, if you set up a clustered index guess where your table goes? It gets moved into the same FILEGROUP as the index! Is this retarded or what! All that planning, all that design, down the toilet. Detached indexes are only valid on non-clustered indexes. ( And people say they will move to SQL if Informix dies. Hee hee! We haven't even talked about logging... ) > Even with Informix on NT (of which i have almost no experience) symlinks are > not available, and further, the recommendations from Informix states that > you can use normal files (O/S buffered and capable of going onto FAT), or > normal files on NTFS (which will be used unbuffered) or a raw partition on > NT. Further, the documentation says that on NT, the use of unbuffered NTFS > files is of equal performance to NT raw spaces, therefore it's not worth > using raw spaces on NT. But that's NT, something I don't play with. > Again with risk. SQL-Server docs specifically point to raw-disks as an unsupported feature, so this is not really an option in Windows. Who would be available to support it? Most Windows people wouldn't even want to attempt this one. What about planning, migrations, etc? There wouldn't be anyone around to admin a raw-disk SQL-Server. :-) ... > > 4) There is no point 4. > > All of these points are significant when major sequential writing is taking > place. > > BTW I must also qualify that this experience applies to machines without > fancy storage managers. If you are using a machine with SCSI disks directly > connected to the SCSI bus or a straight-forward SCSI raid controller, then > you'll notice performance benefits from using raw with Informix (any other > engines?) If it's got a big fat storage manager, then it's implementation > will hide the benefits, in which case, you need to ask "I'm using Engine E > on platform P with storage manager SM, so what's the best storage model to > use?" > I defer to K.I.S.S. > > One other thing you might find attractive to plain ole files > > instead of raw devices is the ability to clone databases. You > > can get quite creative with plain files in ways that you cannot > > with raw devices because of the lock-in of raw-devices. You > > create more work for yourself in the long run with raw disks, > > unless of course you're not as lazy as me and actually enjoy > > the extra work. '-) > > This is where the YMMV slogan comes into play. I would never setup an > Informix,UNIX,SCSI machine with anything else but symlinks and raw spaces. If you have the time be my guest. > If I ever setup a machine with a high performance storage box, I'll look > into the most appropriate mechanism for that. I'd probably setup any brand > of engine on UNIX with symlinks, unless experience or advice shows that it's > pointless. > Welcome to my world. > As for administration of raw spaces, on UNIX it's trivial. Meaningless. Not > a problem. Do what your engine, backup tool and storage manager works best > with. Unless someone else chips in with some detailed advice about other > brands, the original poster will only be lurnin' about Informix engines > today. If the OP wants more advice about Informix, please we should stop > cross-posting and get into more detail only on comp.databases.informix, and > stop boring the other newsgroups. > > Goob, I think you hang around c.d.i quite a bit, so if you wish, please > further my education about the pain of raw spaces on c.d.i. Perhaps some > specific stories are needed so I can undertand your experience. > I'm not arguing that maintaining symlinks or raw-disks are difficult. In an environment where things are somewhat stable and you have the expertise available I'm sure there are benefits. Bu