Slow OS calls on one chip type
Neil Truby reported that dbaccess scripts using shell escapes (e.g. 64 '!echo "Hello World"' lines) ran about 12x slower on a customer's AMD-based RHEL 5.3 servers (~7.8s) than on his own Intel boxes (~0.6s), on identical IDS 11.5FC6; other work like dbimport and index builds was fast, and an IBM PMR was open. Suggestions included terminal type, BIOS/memory/mainboard misconfiguration, comparing a plain shell script, checking top/vmstat/iostat for paging or I/O waits, and using strace/strace -fc to find which call consumed the time. No resolution or root cause is recorded in the thread.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
IDS 11.5FC6 on RHEL 5.3
Here's a starange one. When running SQL which interacts with OS to output
messages, it's very, very much slower at a new customer site than it is at
our own or others. The difference seems to be the AMD processors the
customer has (we have Intel) - the OS and IDS versions are identical.
Here's a a simple test case. Here's some SQL, which echoes out the message
"Hello World" 64 times:
!echo "Hello World"
!echo "Hello World"
!echo "Hello World"
!echo "Hello World"
...
!echo "Hello World"
Run it on the AMD server:
informix@db-1<qa1>:ardenta]$ time dbaccess sysmaster < crap5.sql
Database selected.
Hello World
...
Hello World
Database closed.
real 0m7.776s
user 0m0.016s
sys 0m0.049s
Run it on an Intel server, same IDS version etc:
Hello World
...
Hello World
Database closed.
real 0m0.580s
user 0m0.005s
sys 0m0.037s
12 times as slow on AMD. We noticed it when trunning some data load scripts
which outputs thousands of progress messages, and went from 10m at home to
50m on the client's (far superior) hardware. Of course we could just omit
the statements, but I am worried that this may be part of a bigger issue.
IBM PMR 85722,019,866 refers.
Console or Xwindows terminal? That could be a difference also. FWIW
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art@iiug.org)
See you at the 2010 IIUG Informix Conference
April 25-28, 2010
Overland Park (Kansas City), KS
www.iiug.org/conf
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.
On Thu, Apr 15, 2010 at 6:03 PM, Neil Truby <neil.truby@ardenta.com> wrote:
> IDS 11.5FC6 on RHEL 5.3
>
> Here's a starange one. When running SQL which interacts with OS to output
> messages, it's very, very much slower at a new customer site than it is at
> our own or others. The difference seems to be the AMD processors the
> customer has (we have Intel) - the OS and IDS versions are identical.
>
> Here's a a simple test case. Here's some SQL, which echoes out the message
> "Hello World" 64 times:
> !echo "Hello World"
> !echo "Hello World"
> !echo "Hello World"
> !echo "Hello World"
> ...
> !echo "Hello World"
>
> Run it on the AMD server:
>
> informix@db-1<qa1>:ardenta]$ time dbaccess sysmaster < crap5.sql
>
> Database selected.
>
> Hello World
> ...
> Hello World
>
> Database closed.
>
>
> real 0m7.776s
> user 0m0.016s
> sys 0m0.049s
>
> Run it on an Intel server, same IDS version etc:
>
> Hello World
> ...
> Hello World
>
> Database closed.
>
> real 0m0.580s
> user 0m0.005s
> sys 0m0.037s
>
> 12 times as slow on AMD. We noticed it when trunning some data load
> scripts
> which outputs thousands of progress messages, and went from 10m at home to
> 50m on the client's (far superior) hardware. Of course we could just omit
> the statements, but I am worried that this may be part of a bigger issue.
>
> IBM PMR 85722,019,866 refers.
>
> _______________________________________________
> Informix-list mailing list
> Informix-list@iiug.org
> http://www.iiug.org/mailman/listinfo/informix-list
>
Just some usual suspects:
Misconfigured memory / mainboard parameters in BIOS?
Memory in a slow slot layout (running, but not wrong enough to cause an
error)? Or wrong speed?
Faulty Mainboard? Had this once, caused strange errors, including
intermittent speed behavior. Cause was a chipset chip with faulty
soldering.
Is a C/PERL/bash program also very slow, or only Informix programs?
My personal experience is that AMD // Intel do work more or less within
the same speed range, more problems occur with wrong memory and HD
layout or SAS/SCSI Controllers.
regards
Joerg Volz
------------------------------------------------------------------------
-----
-----Original Message-----
From: informix-list-bounces@iiug.org
[mailto:informix-list-bounces@iiug.org] On Behalf Of Neil Truby
Sent: Friday, April 16, 2010 12:04 AM
To: informix-list@iiug.org
Subject: Slow OS calls on one chip type
IDS 11.5FC6 on RHEL 5.3
Here's a starange one. When running SQL which interacts with OS to
output
messages, it's very, very much slower at a new customer site than it is
at
our own or others. The difference seems to be the AMD processors the
customer has (we have Intel) - the OS and IDS versions are identical.
Here's a a simple test case. Here's some SQL, which echoes out the
message
"Hello World" 64 times:
!echo "Hello World"
!echo "Hello World"
!echo "Hello World"
!echo "Hello World"
...
!echo "Hello World"
Run it on the AMD server:
informix@db-1<qa1>:ardenta]$ time dbaccess sysmaster < crap5.sql
Database selected.
Hello World
...
Hello World
Database closed.
real 0m7.776s
user 0m0.016s
sys 0m0.049s
Run it on an Intel server, same IDS version etc:
Hello World
...
Hello World
Database closed.
real 0m0.580s
user 0m0.005s
sys 0m0.037s
12 times as slow on AMD. We noticed it when trunning some data load
scripts
which outputs thousands of progress messages, and went from 10m at home
to
50m on the client's (far superior) hardware. Of course we could just
omit
the statements, but I am worried that this may be part of a bigger
issue.
IBM PMR 85722,019,866 refers.
_______________________________________________
Informix-list mailing list
Informix-list@iiug.org
http://www.iiug.org/mailman/listinfo/informix-list
IT Handel und Beratung Jorg Volz
Bernhard-Fruh-Str. 7
77855 Achern
GERMANY
Tel: +49 (0)7841-681651
Fax: +49 (0)7841-681654
Mobil: +49 (0)170-2989757
VAT-ID: DE201383541
http://www.it-volz.de
>> "Art Kagel" <art.kagel@gmail.com> wrote in message >> news:mailman.123.1271370641.1071.informix-list@iiug.org... >> Console or Xwindows terminal? That could be a difference also. FWIW I'm using ssh via putty in each case.
"Joerg Volz" <joerg@it-volz.de> wrote in message
news:mailman.124.1271370963.1071.informix-list@iiug.org...
> Just some usual suspects:
>
> Misconfigured memory / mainboard parameters in BIOS?
>
> Memory in a slow slot layout (running, but not wrong enough to cause an
> error)? Or wrong speed?
>
> Faulty Mainboard? Had this once, caused strange errors, including
> intermittent speed behavior. Cause was a chipset chip with faulty
> soldering.
>
> Is a C/PERL/bash program also very slow, or only Informix programs?
>
> My personal experience is that AMD // Intel do work more or less within
> the same speed range, more problems occur with wrong memory and HD
> layout or SAS/SCSI Controllers.
Well I've tried it on two different physical servers at the customer site,
with the same result. So I certainly think a hardware error is unlikely.
It's possible that the same person systematically mis-configured the servers
on assembly.
I might have added that most Informix functions I've tried (dbimport, index
builds) are lightening fast on the AMDs. This system call is the only blip
i've found.
On 15 Apr, 23:54, "Neil Truby" <neil.tr...@ardenta.com> wrote:
> "Joerg Volz" <jo...@it-volz.de> wrote in message
>
> news:mailman.124.1271370963.1071.informix-list@iiug.org...
>
>
>
> > Just some usual suspects:
>
> > Misconfigured memory / mainboard parameters in BIOS?
>
> > Memory in a slow slot layout (running, but not wrong enough to cause an
> > error)? Or wrong speed?
>
> > Faulty Mainboard? Had this once, caused strange errors, including
> > intermittent speed behavior. Cause was a chipset chip with faulty
> > soldering.
>
> > Is a C/PERL/bash program also very slow, or only Informix programs?
>
> > My personal experience is that AMD // Intel do work more or less within
> > the same speed range, more problems occur with wrong memory and HD
> > layout or SAS/SCSI Controllers.
>
> Well I've tried it on two different physical servers at the customer site,
> with the same result. So I certainly think a hardware error is unlikely.
> It's possible that the same person systematically mis-configured the servers
> on assembly.
>
> I might have added that most Informix functions I've tried (dbimport, index
> builds) are lightening fast on the AMDs. This system call is the only blip
> i've found.
- Run a shell script with just the echos in and time that on both.
- Run top on both machines, do they have the same memory free?
- Run vmstat 2 5 and see if paging/swappiing is occuring.
- Run iostat -x 4 and check disk io times.
Otherwise strace the engine and see where it is taking the time ( my
memory is hazy but it could be the misc vp that spawning SYSTEM
calls).
I not sure if this can help , run:
$ strace -fc dbaccess sysmaster < crap5.sql
In both systems and compare what function exactly take more time ....
then you maybe have a way to discovery the reason.
César
--- Em qui, 15/4/10, Neil Truby <neil.truby@ardenta.com> escreveu:
De: Neil Truby <neil.truby@ardenta.com>
Assunto: Slow OS calls on one chip type
Para: informix-list@iiug.org
Data: Quinta-feira, 15 de Abril de 2010, 19:03
IDS 11.5FC6 on RHEL 5.3
Here's a starange one. When running SQL which interacts with OS to output
messages, it's very, very much slower at a new customer site than it is at
our own or others. The difference seems to be the AMD processors the
customer has (we have Intel) - the OS and IDS versions are identical.
Here's a a simple test case. Here's some SQL, which echoes out the message
"Hello World" 64 times:
!echo "Hello World"
!echo "Hello World"
!echo "Hello World"
!echo "Hello World"
...
!echo "Hello World"
Run it on the AMD server:
informix@db-1<qa1>:ardenta]$ time dbaccess sysmaster < crap5.sql
Database selected.
Hello World
...
Hello World
Database closed.
real 0m7.776s
user 0m0.016s
sys 0m0.049s
Run it on an Intel server, same IDS version etc:
Hello World
...
Hello World
Database closed.
real 0m0.580s
user 0m0.005s
sys 0m0.037s
12 times as slow on AMD. We noticed it when trunning some data load scripts
which outputs thousands of progress messages, and went from 10m at home to
50m on the client's (far superior) hardware. Of course we could just omit
the statements, but I am worried that this may be part of a bigger issue.
IBM PMR 85722,019,866 refers.
_______________________________________________
Informix-list mailing list
Informix-list@iiug.org
http://www.iiug.org/mailman/listinfo/informix-list