RE: IDS 10.0 performance on RedHat

Hi, Neil, I have very similar results for Suse Enterprise Linux 9 (with quad dual-core Opteron machines and very good external arrays): overall throughput is better with Informix AIO VP, then with KAIO. Here are some thought about that. First of all, for Informix 10 on Linux 2.6 there is no difference between character and block devices. Starting with kernel 2.6, two new features were introduced in Linux: KAIO and O_DIRECT (direct, non-buffered I/O) When file (or device, block or character) is opened in O_DIRECT mode, it behaves exactly like a raw device. That is, all reads and writes do not go through a kernel buffer cache (in kernel 2.6 called 'page cache'). Informix opens both block and raw devices with O_DIRECT flag. Also, on Linux 2.6, KAIO is supported for raw devices and for files and block devices (!!!), opened with O_DIRECT flag. Informix 10 uses O_DIRECT flag for block and raw devices (not for files :-( ) and uses KAIO for both raw and block devices (again, not for files). I think, Informix developers decided not to use O_DIRECT and KAIO for files just for compatibility with other UNIX flavors. It might be a good feature request to implement O_DIRECT and KAIO for files. As for the KAIO performance, one should distinguish between two different performance goals: - minimize disk access latency; - maximize overall throughput The first measure (latency) is critical for those applications, that access disk sequentially. For databases, the good example is a batch job, that processes data in a cursor for a single table. Another example is 'update statistics low', that can access next leaf index page only after getting it's address from a previous page. The other measure (I/O throughput) is much more critical for parallel environments, like OLTP system with many sessions making transactions in parallel, or even a single session, making 'index to data read ahead's', or parallel index builds... Obviously, in the initial implementation of KAIO in 2.6 Linux kernel, the concentration was on the first goal: traditional I/O subsystem in the Linux kernel is more robust and is capable of handling more request in parallel, especially if I/O device itself is inherently parallel, like EMC 'Clarion' (that is, capable of doing many, up to 255 in case of SCSI, requests in parallel, using it's own powerful CPU) My observation is that for parallel load, Informix AIO gives up to 100% better throughput (I don't use term 'performance' here!), then AIO. Just allocate as many AIO VP's, as you need (30+)! -Alexey > -----Original Message----- > From: informix-list-bounces@iiug.org [mailto:informix-list-bounces@iiug.org] > On Behalf Of Neil Truby > Sent: Wednesday, July 19, 2006 5:24 PM > To: informix-list@iiug.org > Subject: IDS 10.0 performance on RedHat > > Having now got IDS 10.0FC5 to work on RHEL I thought I'd share a few > findings. > > These are based on multipe runs of a dbimport, and of populating a single > large-ish table then building a couple of indexes thereon, utilising PDQ, > thereafter. > > The server is an IBM x3850 with 4 dual-core cpus (these boxes are > *seriously* fast, kids), writing to an EMC Clariion CX300 RAID 1+0. (All > supplied by Ardenta, the UK's premier system integrator ;-) ) > > The baseline time for the dbimport with kaio disabled was 90 mins. With > kaio enabled and 3 cpu vps about 100 mins. With 8 or12 cpu vps down to about > 65 mins, with neither consistently faster than the other. 16 cpu vps no > better than 12. > > Similarly scalable results with the load and index build, with the index > builds increasing appreciably in speed up to the limit of 12. > > Interestingly to me, - although unsurprisngly as my colleague Andy Watson, > inter alia, has previously posted this observation - the kaio speeds are not > affected at all by the use of character over block devices. > > Not a thorough benchmark by any means, but I offer it up in case anyone is > interested. > > regards > -- > Neil Truby t:01932 724027 > Director m:07798 811708 > Ardenta Limited e:neil.truby@ardenta.com > > > _______________________________________________ > Informix-list mailing list > Informix-list@iiug.org > http://www.iiug.org/mailman/listinfo/informix-list

>> My observation is that for parallel load, Informix AIO gives up to 100% better throughput (I don't use term 'performance' here!), then AIO. Just allocate as many AIO VP's, as you need (30+)! Unless I'm mis-understanding you, your results are not similar to mine. With low numbers of cpu vps I found kaio slower than using aio, but with as many or more cpus as cores, that is >= 8, I found kaio superior, even against a test with 127 aio vps.

Alexey Sonkin wrote: > Hi, Neil, > > I have very similar results for Suse Enterprise Linux 9 > (with quad dual-core Opteron machines and very good external arrays): > overall throughput is better with Informix AIO VP, then with KAIO. > > Here are some thought about that. > > First of all, for Informix 10 on Linux 2.6 there is no difference > between > character and block devices. Starting with kernel 2.6, two new features > were introduced in Linux: KAIO and O_DIRECT (direct, non-buffered I/O) > > When file (or device, block or character) is opened in O_DIRECT > mode, it behaves exactly like a raw device. That is, all reads and > writes do not go through a kernel buffer cache (in kernel 2.6 called > 'page cache'). > > Informix opens both block and raw devices with O_DIRECT > flag. Also, on Linux 2.6, KAIO is supported for raw devices and for > files and block devices (!!!), opened with O_DIRECT flag. > Informix 10 uses O_DIRECT flag for block and raw devices > (not for files :-( ) and uses KAIO for both raw and block devices > (again, not for files). I think, Informix developers decided not to > use O_DIRECT and KAIO for files just for compatibility with other > UNIX flavors. It might be a good feature request to implement > O_DIRECT and KAIO for files. > <snip> Are writes actually completed on disk when the write request comes back to the cpu vps as complete? Or are they just complete in kernel buffer cache? Just wondering about an O/S crash and what is actually on disk. With KAIO and O_DIRECT I thought that the write was completed on disk before returning to the process; as opposed to completed in buffer cache.