Re: The stats of "aio" VP on SingleProc machine in KAIO mode

In article <KSN_3.11459$K%.256066@news.tpnet.pl>, =?iso-8859-2?Q?Piotr_Wr=F3blewski?= <ptrwrb@kki.net.pl> wrote: > I am running IDS 7.30.UC2 server on HP-UX10.20. > The machine is a single-processor G50 uccupied by about 30 users, > so the 7.30 server does not run as fast as Online 5.05, > but I must upgrade software. > > In my efforts to make things better I try to eliminate unnecessary > processors/resources. > Among the others, the strange thing I noticed is extremely > large count of i/o done through 'aio' processors, while the > server is run in KAIO mode. > My local guru (he reads this newsgroup, too :)=20 > says almost all i/o traffic should go through > 'kio' channels and near nothing through 'aio', which can be observed via > 'onstat -g iov' (statistics attached below). In fact, four-processor = > K370 > machine is running this way. > > My statistics for G50 says, that millions of disk writes go through = > 'aio', but > when we compare 'onstat -g iov' with 'onstat -g iof', the sum of disk = > writes > (-g iof) is nearly equal the count of 'kio' disk writes (-g iov),=20 > and similarly for disk reads. > > So what the large numbers for 'aio' processors mean?=20 > Are they spurious or are the 'aio' processors doing meaningful work?=20 > Is such a behaviour typical for single-processor machines like mine? > Should I run KAIO on single-processor G50 and leave 1 or 2 NUMAIOVPS > or give up KAIO and rely on many (2 per chunk) NUMAIOVPS ? > > Does the statistics for "spin locks" (see onstat -a below) e.g. > 2 2 1.00 mutex lock, name =3D AIOreq > mean that processes wait for disc i/o operations too long? > > Or maybe my interpretation of these all statistics is bad ? ALmost all of the AIO activity is write with no reads. I was thinking that it was sort activity being written to filesystem space by PSORT_DBTEMP but there are no reads so that cannot be it. Looks like message log activity. Have you had incidents where there was a storm of message being written to the message log (like when there is a lock table overflow) or to /dev/console (as when some poorly written application connects and disconnects repeatedly in a loop instead of connecting and disconnecting outside the loop)? This could explain it. Even with KAIO enabled the AIO VPs are responsible for writing to any cooked devices like the message and console logs. The VERY long waits and #s of I/O per wakeup for each of the AIO VPs also points to the console log, which you have pointed to /dev/console usually a very slow serial device. Art S. Kagel Sent via Deja.com http://www.deja.com/ Before you buy.

Użytkownik Art S. Kagel <kagel@bloomberg.net> w wiadomości do grup dyskusyjnych napisał:81h1hu$uot$1@nnrp1.deja.com... > In article <KSN_3.11459$K%.256066@news.tpnet.pl>, > =?iso-8859-2?Q?Piotr_Wr=F3blewski?= <ptrwrb@kki.net.pl> wrote: > > I am running IDS 7.30.UC2 server on HP-UX10.20. > > The machine is a single-processor G50 uccupied by about 30 users, > > so the 7.30 server does not run as fast as Online 5.05, > > but I must upgrade software. > > > > In my efforts to make things better I try to eliminate unnecessary > > processors/resources. > > Among the others, the strange thing I noticed is extremely > > large count of i/o done through 'aio' processors, while the > > server is run in KAIO mode. > > My local guru (he reads this newsgroup, too :)=20 > > says almost all i/o traffic should go through > > 'kio' channels and near nothing through 'aio', which can be observed > via > > 'onstat -g iov' (statistics attached below). In fact, four-processor = > > K370 > > machine is running this way. > > > > My statistics for G50 says, that millions of disk writes go through = > > 'aio', but > > when we compare 'onstat -g iov' with 'onstat -g iof', the sum of disk > = > > writes > > (-g iof) is nearly equal the count of 'kio' disk writes (-g iov),=20 > > and similarly for disk reads. > > > > So what the large numbers for 'aio' processors mean?=20 > > Are they spurious or are the 'aio' processors doing meaningful > work?=20 > > Is such a behaviour typical for single-processor machines like mine? > > Should I run KAIO on single-processor G50 and leave 1 or 2 NUMAIOVPS > > or give up KAIO and rely on many (2 per chunk) NUMAIOVPS ? > > > > Does the statistics for "spin locks" (see onstat -a below) e.g. > > 2 2 1.00 mutex lock, name =3D AIOreq > > mean that processes wait for disc i/o operations too long? > > > > Or maybe my interpretation of these all statistics is bad ? > > ALmost all of the AIO activity is write with no reads. I was thinking > that it was sort activity being written to filesystem space by > PSORT_DBTEMP but there are no reads so that cannot be it. Looks like > message log activity. Have you had incidents where there was a storm > of message being written to the message log (like when there is a lock > table overflow) or to /dev/console (as when some poorly written > application connects and disconnects repeatedly in a loop instead of > connecting and disconnecting outside the loop)? This could explain it. > Even with KAIO enabled the AIO VPs are responsible for writing to any > cooked devices like the message and console logs. The VERY long waits > and #s of I/O per wakeup for each of the AIO VPs also points to the > console log, which you have pointed to /dev/console usually a very slow > serial device. > > Art S. Kagel > > > Sent via Deja.com http://www.deja.com/ > Before you buy. I do not notice any "message storms" on message log file or console. Message log file has grown up to 130KB since the server is active( 2 weeks), so I don't suspect this kind of server activity as the cause of the problem. My friends running the same software and similiar configurations and applications do not see such a strange aio activity count on their HP G40. My other server K370 runs well, too (but it's a 4-proc machine). All dbspaces, including temporary, are created on raw devices (LVM volumes), temporary dbspaces are pointed in onconfig, so user processes can use them by default. I do not think any significant i/o goes through HPUX filesystems. Should I suspect that this large aio activity is the cause (at least one of) of performance problems? Piotr