RE: EMC vs. ordinary RAID 5

Compare apples with apples. The DELL might have ULTRA 160 SCSII, faster disks. U did not specify. Just set up DELL RAID 10 and compare write sustained performance. {Throughput} Starts a 100 processes to do the same test. {Now you test IO/s, not throughput} Let them all write to different places on the disks. Disable cache and check again. Now unplug one of the drives. Test again. Pug the drive in again and test while the pack is rebuilding a RAID 10, or RAID 5. Everyone always forget about IO/s. This will determine the optimal number of DB spaces to create on one configured disk pack. You must do write and read tests. Writes are supposed to be 3 times slower in RAID 5 than in RAID 10. Please note to always disable write cache on the fisical disks itself. SEAGATE had some software to do this. If you ignore this, the a RAID pack will fail at some stage. Read chapter 1 "An Introduction to Performance Tuning" in the performance tuning guide. Let us know what your results where. That is if you have a couple of days to do all the above tests -----Original Message----- From: Stefan Weideneder [SMTP:stefan@weideneder.de] Sent: Thursday, March 30, 2000 9:13 AM To: informix-list@iiug.org Subject: EMC vs. ordinary RAID 5 Hi all, just because some guys of this wonderfull NG told us that we should avoid RAID 5: Here's the result of an I/O throughput comparison unfortunately made on two different platforms: On an EMC with 2TB disk space, 2GB cache connected via 1 x FDDI 100MByte to a HP box I got a maximum of 70MB/sec throughput ( used raw devices with 64KB block size ). Read 1GB, possibly cached by the controller. I read the data 4 x 6 striped disks, each disk was additionally mirrored ( 48 disks ). On a DELL box with a DELL RAID 5 System ( 170GB disk space on 6 disks 64MB cache ) I got a throughput of 65MB/sec ( used block devices ( 2KB blocksize ) on a RedHat Linux Sytem ). The read operations read 700MB of data ( couldn't be cached by the controller ). On both platforms I started several processes to get the best throughput. 8 processes on the HP-box ( 8 CPUs ). 6 processes on the DELL machine ( 2 CPUs ). For all these tests I used the "dd" command for the I/O operations and "time" to measure the real time. On both systems I loaded a 60GB databases. Because I got some troubles with PDQ on the Linux server, I had to disable it during the index built, but overall it took less time to load the database on the Linux Server. So, please let me know why we should not use RAID 5 systems. I/O settings for the IDS: Because I got the best throughput with 6 processes, I must force the Informix engine to use 6 I/O processes. NUMAIOVPS should be set to at least 6, better would be 7-8 for regular OLTP environments. ( No raw device support on Linux ). Fragmentation strategies: Informix determines the number of scan threads for a sequential scan operation on a single table on the number of table-fragments. Each table-fragment must reside on a different dbspace, therefore you will need 6 dbspaces. By using an expression based fragmentation strategy you can reduce the disk I/O ( the speedup depends on the size of the fragments which can be excluded by the optimizer ). Any comments ? Stefan Phone: +49 89/3565478-2 --------------- --- Fax: +49 89/3565478-3 ------------- ------ mailto:/stefan@weideneder.de --- -------- http://www.weideneder.de -----

Hi Hannes, Hannes Visagie wrote: > > Compare apples with apples. The DELL might have ULTRA 160 SCSII, faster > disks. U did not specify. That's true. I didn't compare apples with apples. I compared a 1,000,000$ system vs. a 20,000$ system. > Just set up DELL RAID 10 and compare write sustained performance. At the moment the RAID 5 System is using a 64MB write cache. BUFFERS is set to 100,000 and LRU_MAX_DIRTY is set to 10%. This ensures that the buffer pool will not contain more than 10,000 modified buffers at a time. The checkpoint duration will never exceed one second ! There's a seperate battery power supply for the case of a power fail. > {Throughput} > Starts a 100 processes to do the same test. Why should I use 100 processes ? There's no Kernel AIO available on the Linux machine and therefore read/write requests will be handled by the AIO-vps. As I mentioned, six AIO-vps worked better than 7 AIO-vps. {Now you test IO/s, not > throughput} Let them all write to different places on the disks. Generally, user-threads do not write to disks. Modifications are done inside the buffer pool and are not written directly to the disk. Before you can update or modify a row, you must read the row from the disk. If I would need sorted writes, I would force chunk writes ( checkpoint ). That's one of the great Informix features. If all the users would read from different places on the disk, and if these pages can't be found in the buffer pool, I would have a 0% cached read output ( onstat -p ). I hope that you have never seen such an environment ! > Disable cache and check again. Why shouldn't I use what I have paid for ? > Now unplug one of the drives. Test again. Pug the drive in again and test > while the pack is rebuilding a RAID 10, or RAID 5. Most time will be spent for the time it will take to plug in the new disk :-) > Everyone always forget about IO/s. This will determine the optimal number > of DB spaces to create on one configured disk pack. Can you explain this in more detail, please ? > You must do write and read tests. Writes are supposed to be 3 times slower > in RAID 5 than in RAID 10. That's true if you do not use the write cache ! > > Please note to always disable write cache on the fisical disks itself. > SEAGATE had some software to do this. If you ignore this, the a RAID pack > will fail at some stage. And do not forget that there are a lot of other circumstances that might cause disk problems. A good method is to use a second instance on a second server. > Read chapter 1 "An Introduction to Performance Tuning" in the performance > tuning guide. I'm not the author of this book. And I'm sure you have never read my checklist ! > Let us know what your results where. That is if you have a couple of days > to do all the above tests I will not run all the tests, because there is no reason to improve the disk speed on a machine with just 2 CPUs. As you know it is impossible for our database server running on a 2 processor machine to process more than 65MB data input per second. And I never intended to procduce a benchmark test. I'm just interested in real environments ! Thanks a lot, Stefan > -----Original Message----- > From: Stefan Weideneder [SMTP:stefan@weideneder.de] > Sent: Thursday, March 30, 2000 9:13 AM > To: informix-list@iiug.org > Subject: EMC vs. ordinary RAID 5 > > Hi all, > > just because some guys of this wonderfull NG told us > that we should avoid RAID 5: > > Here's the result of an I/O throughput comparison > unfortunately made on two different platforms: > > On an EMC with 2TB disk space, 2GB cache connected > via 1 x FDDI 100MByte to a HP box I got a maximum of > 70MB/sec throughput ( used raw devices with 64KB > block size ). Read 1GB, possibly cached by the > controller. I read the data 4 x 6 striped disks, > each disk was additionally mirrored ( 48 disks ). > > On a DELL box with a DELL RAID 5 System ( 170GB disk space > on 6 disks 64MB cache ) I got a throughput of > 65MB/sec ( used block devices ( 2KB blocksize ) > on a RedHat Linux Sytem ). The read operations > read 700MB of data ( couldn't be cached by the > controller ). > > On both platforms I started several processes > to get the best throughput. 8 processes on the > HP-box ( 8 CPUs ). 6 processes on the DELL > machine ( 2 CPUs ). For all these tests I used > the "dd" command for the I/O operations and > "time" to measure the real time. > > On both systems I loaded a 60GB databases. Because > I got some troubles with PDQ on the Linux server, > I had to disable it during the index built, but > overall it took less time to load the database > on the Linux Server. > So, please let me know why we should not use > RAID 5 systems. > > I/O settings for the IDS: > > Because I got the best throughput with 6 processes, > I must force the Informix engine to use 6 I/O processes. > NUMAIOVPS should be set to at least 6, better would be > 7-8 for regular OLTP environments. ( No raw device support > on Linux ). > > Fragmentation strategies: > > Informix determines the number of scan threads for a > sequential scan operation on a single table on the number of > table-fragments. Each table-fragment must reside on a > different dbspace, therefore you will need 6 dbspaces. > > By using an expression based fragmentation strategy > you can reduce the disk I/O ( the speedup depends on > the size of the fragments which can be excluded by > the optimizer ). > > Any comments ? > > Stefan > > Phone: +49 89/3565478-2 --------------- > --- Fax: +49 89/3565478-3 ------------- > ------ mailto:/stefan@weideneder.de --- > -------- http://www.weideneder.de -----

Hi: Have you looked at the performance numbers of the NetApp Filers? http://www.zerowait.com/filers.html Hannes Visagie wrote: > Compare apples with apples. The DELL might have ULTRA 160 SCSII, faster > disks. U did not specify. > > Just set up DELL RAID 10 and compare write sustained performance. > {Throughput} > Starts a 100 processes to do the same test. {Now you test IO/s, not > throughput} Let them all write to different places on the disks. > Disable cache and check again. > > Now unplug one of the drives. Test again. Pug the drive in again and test > while the pack is rebuilding a RAID 10, or RAID 5. > > Everyone always forget about IO/s. This will determine the optimal number > of DB spaces to create on one configured disk pack. > > You must do write and read tests. Writes are supposed to be 3 times slower > in RAID 5 than in RAID 10. > > Please note to always disable write cache on the fisical disks itself. > SEAGATE had some software to do this. If you ignore this, the a RAID pack > will fail at some stage. > > Read chapter 1 "An Introduction to Performance Tuning" in the performance > tuning guide. > > Let us know what your results where. That is if you have a couple of days > to do all the above tests > > -----Original Message----- > From: Stefan Weideneder [SMTP:stefan@weideneder.de] > Sent: Thursday, March 30, 2000 9:13 AM > To: informix-list@iiug.org > Subject: EMC vs. ordinary RAID 5 > > Hi all, > > just because some guys of this wonderfull NG told us > that we should avoid RAID 5: > > Here's the result of an I/O throughput comparison > unfortunately made on two different platforms: > > On an EMC with 2TB disk space, 2GB cache connected > via 1 x FDDI 100MByte to a HP box I got a maximum of > 70MB/sec throughput ( used raw devices with 64KB > block size ). Read 1GB, possibly cached by the > controller. I read the data 4 x 6 striped disks, > each disk was additionally mirrored ( 48 disks ). > > On a DELL box with a DELL RAID 5 System ( 170GB disk space > on 6 disks 64MB cache ) I got a throughput of > 65MB/sec ( used block devices ( 2KB blocksize ) > on a RedHat Linux Sytem ). The read operations > read 700MB of data ( couldn't be cached by the > controller ). > > On both platforms I started several processes > to get the best throughput. 8 processes on the > HP-box ( 8 CPUs ). 6 processes on the DELL > machine ( 2 CPUs ). For all these tests I used > the "dd" command for the I/O operations and > "time" to measure the real time. > > On both systems I loaded a 60GB databases. Because > I got some troubles with PDQ on the Linux server, > I had to disable it during the index built, but > overall it took less time to load the database > on the Linux Server. > So, please let me know why we should not use > RAID 5 systems. > > I/O settings for the IDS: > > Because I got the best throughput with 6 processes, > I must force the Informix engine to use 6 I/O processes. > NUMAIOVPS should be set to at least 6, better would be > 7-8 for regular OLTP environments. ( No raw device support > on Linux ). > > Fragmentation strategies: > > Informix determines the number of scan threads for a > sequential scan operation on a single table on the number of > table-fragments. Each table-fragment must reside on a > different dbspace, therefore you will need 6 dbspaces. > > By using an expression based fragmentation strategy > you can reduce the disk I/O ( the speedup depends on > the size of the fragments which can be excluded by > the optimizer ). > > Any comments ? > > Stefan > > Phone: +49 89/3565478-2 --------------- > --- Fax: +49 89/3565478-3 ------------- > ------ mailto:/stefan@weideneder.de --- > -------- http://www.weideneder.de ----- -- Michael Linett - President Mike@zerowait.com * http://www.zerowait.com PH 302.266.9408 FX 302.738.4302 Superior High Availability Solutions