Page cleaners and raids

Hi Y'all, We are running Online v5.10 and our data is stored on RAID 1 Can someone tell me whether the number of page cleaners is significant in a Raid environment.and I know standard advice is to move the locical logs out of the root db space, but again, does this make any difference in a RAID environment ? Thanks Debbie

Debbie Caswell wrote in message <923605DC7F0FD311BD8C0090274D146D17B91E@ccwin02.cclub.uk.com>... > >We are running Online v5.10 and our data is stored on RAID 1 >Can someone tell me whether the number of page cleaners is significant >in a Raid environment.and I know standard advice is to move the locical >logs out of the root db space, but again, does this make any difference >in a RAID environment ? Hmmmm - RAID 1 - that's hardware mirroring, isn't it? Or is it striping? No, mirroring. All the rules for cleaners, disk activity, etc apply to such systems, however you need to pretend you have 1/2 the number of disks you really have. If you've been following the CLEANERS thread and seen the pfread process mentioned, you can correctly apply that test also to the logical device name that represents the mirrored pair. Some would argue that using Informix mirroring and generally exploiting smart layouts can give you better performance than using hardware, but that depends on the smartness and speed of your RAID controller vs. the smartness of your layouts. On that subject, (and sorry to diverge from what you may understand, Debbie) can someone clear up a question I've been wanting to ask? With the split-reads offered by Informix mirroring, does it split the addresses thru the middle of the total chunk size, or thru the middle of the high-watermark of the currently allocated space in the chunk? Seems to be it should take the watermark into consideration, cos if you allocate a 1G space but currently only load 200M, then you ain't gonna get split reads. __END__ Andrew Hamm Technical Consultant Sanderson Australia Pty Ltd e-mail: <mailto:ahamm@sanderson.net.au> web: <http://www.sanderson.net.au>; -- I like cats too - let's exchange recipies

Thanks for that info Andrew, However, I've just spoken to my sys admin person and she has informed me that we're not actually using a raid, she has manually striped and mirrored the discs (isn't that all a raid does anyway ???) This is transparent to the informix system, so back to the second part of my original question, is it worth taking the logical logs out of the rootdbs ? Debbie > -----Original Message----- > From: Andrew Hamm [mailto:ahamm@sanderson.net.au] > Posted At: 14 December 2000 00:22 > Posted To: informix > Conversation: Page cleaners and raids > Subject: Re: Page cleaners and raids > > > Debbie Caswell wrote in message > <923605DC7F0FD311BD8C0090274D146D17B91E@ccwin02.cclub.uk.com>... > > > >We are running Online v5.10 and our data is stored on RAID 1 > >Can someone tell me whether the number of page cleaners is > significant > >in a Raid environment.and I know standard advice is to move > the locical > >logs out of the root db space, but again, does this make any > difference > >in a RAID environment ? > > Hmmmm - RAID 1 - that's hardware mirroring, isn't it? Or is > it striping? No, > mirroring. > > All the rules for cleaners, disk activity, etc apply to such systems, > however you need to pretend you have 1/2 the number of disks > you really > have. If you've been following the CLEANERS thread and seen the pfread > process mentioned, you can correctly apply that test also to > the logical > device name that represents the mirrored pair. > > Some would argue that using Informix mirroring and generally > exploiting > smart layouts can give you better performance than using > hardware, but that > depends on the smartness and speed of your RAID controller > vs. the smartness > of your layouts. > > On that subject, (and sorry to diverge from what you may > understand, Debbie) > can someone clear up a question I've been wanting to ask? With the > split-reads offered by Informix mirroring, does it split the > addresses thru > the middle of the total chunk size, or thru the middle of the > high-watermark > of the currently allocated space in the chunk? Seems to be it > should take > the watermark into consideration, cos if you allocate a 1G space but > currently only load 200M, then you ain't gonna get split reads. > > __END__ > Andrew Hamm > Technical Consultant > Sanderson Australia Pty Ltd > e-mail: <mailto:ahamm@sanderson.net.au> > web: <http://www.sanderson.net.au>; > -- > I like cats too - let's exchange recipies > > >

I've had a few replies from people which have made me realise I totally failed to address your question on moving the logical logs out of the rootdbs. Sorry! Also, thanks for the questions - it's giving me concrete things to study and justify. Since tuning has become my main job for the next few months, it's not just an excellent opportunity to do better, but a chance to practice explaining and justifying my theories, which is one of the hardest things in sysadmin. Keep 'em coming! Including contradictions and improvements from anyone. On with the show... Debbie Caswell wrote in message <923605DC7F0FD311BD8C0090274D146D17D21C@ccwin02.cclub.uk.com>... > >I've just spoken to my sys admin person and she has informed me >that we're not actually using a raid, she has manually striped and >mirrored the discs (isn't that all a raid does anyway ???) > Software raid sounds like what you've got vs. hardware raid. Software raid is generally slagged off as low performance and to be avoided. Informix mirroring, separate write threads and split reads should out-perform any software raid system. The major complaint about software raid is that it steals a bunch of CPU cycles that could be better applied elsewhere. After all, Informix does it's own disk scheduling that basically makes the engine a mini-operating system of it's own, so why do double work? But I'd be really interested in any contradictory experiences from anyone, backed up with numbers please. I'm inclined to avoid both types of raid but I'm looking for definitive numbers or guidelines to either justify or shatter my illusions. Informix claim to have a white-paper on RAID but I've never managed to get hold of it. Testing is really tricky 'cos I think the only worthwhile tests are live systems, and who wants to let you piss around with a critical business system? Experimentation is slow... Any kind of raid system presents a bunch of disks as one, so whenever I talk about a disk spindle or a disk, you can take that to mean a raided logical disk too. I think some raid systems not only glue multiple disks together but then can slice the result across and present it as two or more logical disks. Even non-RAID machinery can easily slice thru disks and present the parts to the O/S as if they were separate disks. Whenever you have disks being sliced this way, you are then going to have to consider all of those slices as part of one disk when it comes to counting threads. For eg, SCO is a classic here since it has fairly limited partitioning. A SCO disk can be split into 4 DOS-style partitions - one of the artifacts of the ancient DOS partitioning rules that infects Intel PC's generally. So, if you buy a 20G disk, then install SCO with the disk allocated as one large partition, SCO then makes you split that into "divisions" - upto 7. 5 divisions are instantly taken by the O/S, leaving you with 2. Since Online has a 2G limit to chunks, you are not going to be able to use all the remaining space. So instead of one partition, you allocate 2 or more partitions and then you can have 9, 16, or 23 divisions to spread out the chunks on the physical disk. The partitions means that SCO can "see" 2 3 or 4 devices that it thinks are physical disks. HOWEVER - and this warning applies to any system that slices disks - they are still on one physical disk so they affect the total thread throughput of the disk. What I'm trying to say is - take into account when disks are glued together by RAID, and beware of when disks (raided or not) are cut into several slices. Apply the pfread performance test to one (possibly raid bundled) disk and know that it applies to all the slices taken of that device. Linux offers a few more partitioning options apart from having to suffer the 4 DOS partitioning limitation, but it's got it's own peculiarities - just like every other UNIX in existence. >This is transparent to the informix system, so back to the second part >of my original question, is it worth taking the logical logs out of the >rootdbs ? > Logs are typically moved out of the rootdbs immediately after you start an engine - to reduce the number of writes to the rootdbs and minimise contention for critical and highly-useful internal data structures that are stored in rootdbs. It may also reduce the chances of rootdbs's destruction if there is less total activity going on there. I'm talking about hardware glitches etc which strike randomly on rare occasions. The less write activity there, the less chance for damage. But this is talking about a relatively slim chance of disaster. One good trick with logs is to put alternative ones onto different disks. Eg 1, 3, 5, 7, ... onto one disk, and then 2, 4, 6, 8, ... onto another disk. The reason being, when a log becomes full, the next log starts to get written into, and the last log starts to get transferred to tape (assuming you are using log tapes!) and so you want to separate those two activities which WILL happen together. This theory does not stretch to sharing around 3, 4, 5, ... disks. It's pointless just for this reason alone. Physical log should also be shifted out from the root space, and it's a good candidate for placement onto a 3rd spindle that is NOT used for logical logs. If your production measurements prove that log activity is a bottleneck, then there may be some justification for striping the beasts in hardware, despite a decision to avoid raid elsewhere. But only future measurements will give you enough hindsight. I haven't seen any measurements that would justify this with our customers, but I'll keep an eye on this. Traditional Informix suggestions is for total logging sizes to be 20% of total database space - don't forget to take future growth into account. If your DB will be 50G in 5 years, then make the logs 10G. That seems like a huge number but what the hey. I've never had a problem from going 20%. The main theory from Informix is that it's a good amount of space to support online archiving of an active database without running out of log resources due to the blocking of certain things during archives. If you have a 50G db then surely there will be some big activity going on. That's the theory anyway. One day in the future, especially if you are desperate to find some space, you will have had a chance to get very familiar with the rate of log consumption on the busiest days and you could reduce the sizes of the logs if you really insisted. Of that space, split the phys and logicals 1:3 - so 10G => 2.5G physical, and 7.5G logical. That seems like a huge number but what the hey.... Split the logs down into decent sized pieces. I tend to go for a split that helps to make the logs turn over on a fairly regular hourly basis or faster, but with logs this large you may be struggling*. If so, reduce the total log size. I'm starting to be very dubious of this 20% allocation these days with disks being so big and so much of our data being historical crap that is mostly ignored, but I admit to not having measured the subject for a long time. This rule, however, got around the failing archives that I've mentioned recently in a different thread involving tomatoes. * Monitoring log roll-over will tell you how much you will lose if you have a ca

Same thing. RAID is just a formalization of the definitions of various protection and performance configurations so you do indeed have RAID. The question for me is whether your administrator has configured RAID01 or RAID10. Performance wise they are equivalent but for protection and recovery a RAID10 configuration (striped mirrored pairs [or triplets]) is superior to a RAID01 configuration (mirrored stripe sets). Art S. Kagel Debbie Caswell wrote: > > Thanks for that info Andrew, > > However, I've just spoken to my sys admin person and she has informed me > that we're not actually using a raid, she has manually striped and > mirrored the discs (isn't that all a raid does anyway ???) > > This is transparent to the informix system, so back to the second part > of my original question, is it worth taking the logical logs out of the > rootdbs ? > > Debbie [SNIP]