DISK ARRAY
A DBA asked how to carve up a 310GB EMC Clarion RAID-1 array for Informix: the hardware vendor advised striping everything across all spindles, while traditional Informix advice is to separate critical, data and index dbspaces. No single answer emerged, but most replies favoured striping on such arrays. Points made: chunks were limited to 2GB, so many logical devices must be presented; one poster warned to ignore non-Informix tuning advice and expose spindles directly; others reported EMC cache makes manual placement pointless, that random striping suits OLTP (with separate fragmentation for very large tables, DSS differing), and Art Kagel recommended "plaiding" RAID10 sets with a small (16K) stripe size plus fragmentation where fragment elimination/parallel scans help.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Has anybody had any experience with a large disk array? We have a EMC Clarion disk array with 310 GB of disk space which is setup with Hardware Raid 1 (Fully mirrored). I am getting conflicting information on how to dice up the disk space. Hardware vendor says to strip all logical volumes across all disk drives in the system. Informix (and my experience) says to separate everything possible (Critical spaces from data spaces, data spaces from indexes spaces..). I was just wondering what has been successful for anyone who has done this. Thanks in advance! Ronald Twaddell, DBA Nexigent ronald.twaddell@nexigent.com (317)275-2330 (317)332-5877 - Cell Phone (317)275-2301 - FAX ----------------------------------------------------- Confidentiality Notice: This e-mail transmission may contain confidential or legally privileged information that is intended only for the individual or entity named in the e-mail address. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reliance upon the contents of this e-mail is strictly prohibited. If you have received this e-mail transmission in error, please reply to the sender, so that Nexigent can arrange for proper delivery, and then please delete the message from your inbox. Thank you.
Twaddell, Ronald wrote in message <94id5p$9f8$1@news.xmission.com>... > >Has anybody had any experience with a large disk array? > >We have a EMC Clarion disk array with 310 GB of disk space which is setup >with Hardware Raid 1 (Fully mirrored). I am getting conflicting information >on how to dice up the disk space. Hardware vendor says to strip all logical >volumes across all disk drives in the system. Informix (and my experience) >says to separate everything possible (Critical spaces from data spaces, data >spaces from indexes spaces..). I was just wondering what has been >successful for anyone who has done this. > Firstly, unless you are using a particularly hard-core port of the engine (which I don't even know if they exist yet), most or all current versions of the engine are limited to 2GB chunks. So you'll have to present 310 / 2 / 2 = ??emm-you-work-it-out?? devices to the O/S. Performance is better with informix if you can expose the disk spindles directly - that allows you to carefully spread the load out onto all physical disks. A striping system does not make it easy to control that, but it does at least give you some disk parallelism if you have a totally amorphous blob otherwise known as a "one-in-all-in" disk allocation - ie all the tables thrown in with sod-all thought to decent locations. Rule number one of disk allocation: TOTALLY IGNORE THE ADVICE OF HARDWARE VENDORS OR PEOPLE WHO TUNE UNIX OR OTHER DATABASES SUCH AS ORACLE. The rules are almost but not quite completely different for Informix because it uses raw spaces and totally manages it's own scheduling algorithms and buffering from the disks. That fact alone invalidates just about all other advice on the subject that is not from the mouth of an Informix guru. So, now you know you can safely ignore the hardware vendor...
I use such an array. As Anndrew Hamm said, you'll have to present the 9 or 18GByte logical units you see from EMC as 2 GByte chunks. So, there will be an inherent level of indirection between the way you see the chunks on Informix and their physical position. In my experience, the disks are so fast, and the effect of the EMC cache so great, that i/o service times are very low indeed, typically 8ms below. I have therefore seen little benefit to be had from pursuing a laborious disk positioning strategy, and paid little attention to the physical location of my chunks. I have seen nothing to convince me that this is a flawed approach *on ny systems*. Twaddell, Ronald <Ronald.Twaddell@nexigent.com> wrote in message news:94id5p$9f8$1@news.xmission.com... > > Has anybody had any experience with a large disk array? > > We have a EMC Clarion disk array with 310 GB of disk space which is setup > with Hardware Raid 1 (Fully mirrored). I am getting conflicting information > on how to dice up the disk space. Hardware vendor says to strip all logical > volumes across all disk drives in the system. Informix (and my experience) > says to separate everything possible (Critical spaces from data spaces, data > spaces from indexes spaces..). I was just wondering what has been > successful for anyone who has done this. > > Thanks in advance! > > > Ronald Twaddell, DBA > Nexigent > ronald.twaddell@nexigent.com > (317)275-2330 > (317)332-5877 - Cell Phone > (317)275-2301 - FAX > > > > > ----------------------------------------------------- > Confidentiality Notice: This e-mail transmission > may contain confidential or legally privileged > information that is intended only for the individual > or entity named in the e-mail address. If you are not > the intended recipient, you are hereby notified that > any disclosure, copying, distribution, or reliance > upon the contents of this e-mail is strictly prohibited. > > If you have received this e-mail transmission in error, > please reply to the sender, so that Nexigent can arrange > for proper delivery, and then please delete the message > from your inbox. Thank you.
Neil Truby wrote in message <94juea$rmd$1@lyonesse.netcom.net.uk>... >I use such an array. > >In my experience, the disks are so fast, and the effect of the EMC cache so >great, that i/o service times are very low indeed, typically 8ms below. I >have therefore seen little benefit to be had from pursuing a laborious disk >positioning strategy, and paid little attention to the physical location of >my chunks. > >I have seen nothing to convince me that this is a flawed approach *on ny >systems*. > Possibly so. I read yesterday that ATA disks are now out-performing FW SCSI and offering bigger capacities. This industry changes way too fast. How wide are your stripes? Surely 2 is insufficient, but has anyone got a rule of thumb or numbers showing a decent stripe width that's really effective? PS - how do you take advantage of parallelism and fragmentation if your disk system is behaving like one or two spindles?
If it's an OLTP system, just mindlessly striping over all (or half if you mirror) disks is probably best. Typically <2% of all reads will turn into disk accesses, and they'll tend to be fairly random. By essentially "trying" to make the spot you touch on disk more random as well (with striping), you virtually guarantee nearly even workload balance over your disks. It is also very tolerant to changes in access/workload patterns. The only way to improve on this would be to try to produce balanced disk hits in a non random fashion (ie more sequential) which is a ton of work and a losing game. Not worth the cost. If you have a few very large tables (>5-10GB each), you might consider a separate fragmentation scheme for them. This hybrid approach is suggested in an SAP/Informix white paper from the "Terabyte Project", which makes recommendations for operating such very large OLTP type systems. You might search SAP's website, it may be available there. Interesting reading. DSS is different, since you know in advance you will be doing a lot of sequential scanning. Greg "Twaddell, Ronald" wrote: > Has anybody had any experience with a large disk array? > > We have a EMC Clarion disk array with 310 GB of disk space which is setup > with Hardware Raid 1 (Fully mirrored). I am getting conflicting information > on how to dice up the disk space. Hardware vendor says to strip all logical > volumes across all disk drives in the system. Informix (and my experience) > says to separate everything possible (Critical spaces from data spaces, data > spaces from indexes spaces..). I was just wondering what has been > successful for anyone who has done this. > > Thanks in advance! > > Ronald Twaddell, DBA > Nexigent > ronald.twaddell@nexigent.com > (317)275-2330 > (317)332-5877 - Cell Phone > (317)275-2301 - FAX > > ----------------------------------------------------- > Confidentiality Notice: This e-mail transmission > may contain confidential or legally privileged > information that is intended only for the individual > or entity named in the e-mail address. If you are not > the intended recipient, you are hereby notified that > any disclosure, copying, distribution, or reliance > upon the contents of this e-mail is strictly prohibited. > > If you have received this e-mail transmission in error, > please reply to the sender, so that Nexigent can arrange > for proper delivery, and then please delete the message > from your inbox. Thank you.
EMC disks are fast as long as mostly cache is used for i/o. I don't know EMC Clarion, but in EMC Symmetrix (thats what we use) each logical disk (for example 9GB) has its own cache. So, when in normal operating some disks have lots of i/o, and many other disks are doing nothing, you are 'wasting' lots of EMC-Cache. We had this situation with our 160GB SAP R/3 database. When changing from Reliant Unix Platform to Solaris Platform, we therefore striped our disks. A good database layout should be the best, if you can achieve, that all logical EMC disks have nearly the same i/o. But with a SAP-database, striping is a good choice. Markus "Twaddell, Ronald" schrieb: > > Has anybody had any experience with a large disk array? > > We have a EMC Clarion disk array with 310 GB of disk space which is setup > with Hardware Raid 1 (Fully mirrored). I am getting conflicting information > on how to dice up the disk space. Hardware vendor says to strip all logical > volumes across all disk drives in the system. Informix (and my experience) > says to separate everything possible (Critical spaces from data spaces, data > spaces from indexes spaces..). I was just wondering what has been > successful for anyone who has done this. > > Thanks in advance! > > Ronald Twaddell, DBA
I say take all those mirrored pairs, create 3 or 4 RAID10 stripe sets out of them and then Plaid the disks by striping the 4 RAID10's into one HUGE stripe (make sure the stripe size is set small 16K is best for Informix). Now chop up the one 310GB logical device into 155 2GB logical drives and build your engine. If you have large tables that can take advantage of fragment elimination or parallel scanning then by all means also fragment the tables, otherwise rely on the EMCs to do the load balancing. That's what I do and thing fly. Art S. Kagel "Twaddell, Ronald" wrote: > > Has anybody had any experience with a large disk array? > > We have a EMC Clarion disk array with 310 GB of disk space which is setup > with Hardware Raid 1 (Fully mirrored). I am getting conflicting information > on how to dice up the disk space. Hardware vendor says to strip all logical > volumes across all disk drives in the system. Informix (and my experience) > says to separate everything possible (Critical spaces from data spaces, data > spaces from indexes spaces..). I was just wondering what has been > successful for anyone who has done this. > > Thanks in advance! > > Ronald Twaddell, DBA > Nexigent > ronald.twaddell@nexigent.com > (317)275-2330 > (317)332-5877 - Cell Phone > (317)275-2301 - FAX > > ----------------------------------------------------- > Confidentiality Notice: This e-mail transmission > may contain confidential or legally privileged > information that is intended only for the individual > or entity named in the e-mail address. If you are not > the intended recipient, you are hereby notified that > any disclosure, copying, distribution, or reliance > upon the contents of this e-mail is strictly prohibited. > > If you have received this e-mail transmission in error, > please reply to the sender, so that Nexigent can arrange > for proper delivery, and then please delete the message > from your inbox. Thank you.