Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower
Posted in 2005
Steve ran IDS 9.40.FC5 on three Alpha Tru64 5.1B boxes and found his newest, fastest GS1280 ('Royale') performed worse as a database server than a smaller 4-CPU ES45, with multi-second stalls where all apps (even dirty reads) froze. He reported identical SAN disks/database, same ONCONFIG apart from NUMCPUVPS, KAIO on, good cache rates, and forced full-duplex NICs. Suggestions included SAN/disk contention, update statistics, reducing NUMCPUVPS, duplex/autonegotiation mismatches, checking loopback/connection settings (sqlhosts, hosts, onstat -g ntt) and enabling WSTATS to examine sysseswts wait reasons. No resolution is recorded in the thread.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Transactions, Locking & Isolation, Logging & Checkpoints, Versions, Editions & End-of-Life
We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems with Unix 5.1B. Royale: GS1280 8 - 1.3 EV7 CPU's 8 - gig of memory Broadhurst: ES80 8 - 1.0 EV7 CPU's 8 - gig of memory Majestic: ES45 4 - 1.25 EV6 CPU's 8 - gig of memory We have a multi-tier in house application so the database never has more than 95 connections. We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu). The systems are connected with a gigabit switch. When using different configurations, the resulting throughput speed makes no sense to us. 1) Royale application server - Majestic db server - fastest combo 2) Broadhurst application server - Majestic db server - 2nd fastest 3) Royale application server AND db server - much slower than the previous 2 configurations 4) Broadhurst application server - Royale db server - slower than configuration #3 At all times the systems are at least 50% idle and usually 75% idle when volume is heavy and timing tests were done. All timings done with the exact same physical disks (2 spindles in a SAN we configure to the db server). We would have thought that configuration 3 would be by far the fastest. We notice 'delays' when it appears that IDS completely stops for 4 or 5 seconds with configuration 4, but only 1 second with configuration 3. 'Delay' is defined as all our applications stop (even dirty read only) like they are all waiting for the db. There are more delays than checkpoints. It appears to us that Royale runs slower and communicates slower than Majestic when it is a db server but is much faster than Majestic as an application server. Does anyone have any ideas why our fastest, newest computer Royale would run slower than the 4 cpu with slower CPU's? Wondering why we bought 'Royale'????? Steve Willcoxon Shubert Organization
"All timings done with the exact same physical disks (2 spindles in a SAN we configure to the db server)." Is it a SAN box where "you don't need to know" (insert typical SAN vendor response here...) which physical disk you are really using? EMC? Bob Roussey Spirit Airlines Informix/Unix Administration Phone: 586.741.8991 e-mail: RobertRo@SpiritAir.com -----Original Message----- From: forum.subscriber@iiug.org [mailto:forum.subscriber@iiug.org] On Behalf Of Steve Willcoxon Sent: Thursday, June 23, 2005 10:55 AM To: ids@iiug.org Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233] We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems with Unix 5.1B. Royale: GS1280 8 - 1.3 EV7 CPU's 8 - gig of memory Broadhurst: ES80 8 - 1.0 EV7 CPU's 8 - gig of memory Majestic: ES45 4 - 1.25 EV6 CPU's 8 - gig of memory We have a multi-tier in house application so the database never has more than 95 connections. We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu). The systems are connected with a gigabit switch. When using different configurations, the resulting throughput speed makes no sense to us. 1) Royale application server - Majestic db server - fastest combo 2) Broadhurst application server - Majestic db server - 2nd fastest 3) Royale application server AND db server - much slower than the previous 2 configurations 4) Broadhurst application server - Royale db server - slower than configuration #3 At all times the systems are at least 50% idle and usually 75% idle when volume is heavy and timing tests were done. All timings done with the exact same physical disks (2 spindles in a SAN we configure to the db server). We would have thought that configuration 3 would be by far the fastest. We notice 'delays' when it appears that IDS completely stops for 4 or 5 seconds with configuration 4, but only 1 second with configuration 3. 'Delay' is defined as all our applications stop (even dirty read only) like they are all waiting for the db. There are more delays than checkpoints. It appears to us that Royale runs slower and communicates slower than Majestic when it is a db server but is much faster than Majestic as an application server. Does anyone have any ideas why our fastest, newest computer Royale would run slower than the 4 cpu with slower CPU's? Wondering why we bought 'Royale'????? Steve Willcoxon Shubert Organization
Compaq Storage Works Raw chunks mapped with LSM on actual known physical disks. Disks not striped (mirrored by SAN so actually 4 spindles) or used by any other system. Reads cached 99+% and writes cached 96+% Steve Willcoxon ----- Original Message ----- From: "Robert Roussey" <RobertRo@SpiritAir.com> To: "Steve Willcoxon" <stevew@shubertorg.com>; <ids@iiug.org> Sent: Thursday, June 23, 2005 11:59 AM Subject: RE: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233] "All timings done with the exact same physical disks (2 spindles in a SAN we configure to the db server)." Is it a SAN box where "you don't need to know" (insert typical SAN vendor response here...) which physical disk you are really using? EMC? Bob Roussey Spirit Airlines Informix/Unix Administration Phone: 586.741.8991 e-mail: RobertRo@SpiritAir.com -----Original Message----- From: forum.subscriber@iiug.org [mailto:forum.subscriber@iiug.org] On Behalf Of Steve Willcoxon Sent: Thursday, June 23, 2005 10:55 AM To: ids@iiug.org Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233] We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems with Unix 5.1B. Royale: GS1280 8 - 1.3 EV7 CPU's 8 - gig of memory Broadhurst: ES80 8 - 1.0 EV7 CPU's 8 - gig of memory Majestic: ES45 4 - 1.25 EV6 CPU's 8 - gig of memory We have a multi-tier in house application so the database never has more than 95 connections. We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu). The systems are connected with a gigabit switch. When using different configurations, the resulting throughput speed makes no sense to us. 1) Royale application server - Majestic db server - fastest combo 2) Broadhurst application server - Majestic db server - 2nd fastest 3) Royale application server AND db server - much slower than the previous 2 configurations 4) Broadhurst application server - Royale db server - slower than configuration #3 At all times the systems are at least 50% idle and usually 75% idle when volume is heavy and timing tests were done. All timings done with the exact same physical disks (2 spindles in a SAN we configure to the db server). We would have thought that configuration 3 would be by far the fastest. We notice 'delays' when it appears that IDS completely stops for 4 or 5 seconds with configuration 4, but only 1 second with configuration 3. 'Delay' is defined as all our applications stop (even dirty read only) like they are all waiting for the db. There are more delays than checkpoints. It appears to us that Royale runs slower and communicates slower than Majestic when it is a db server but is much faster than Majestic as an application server. Does anyone have any ideas why our fastest, newest computer Royale would run slower than the 4 cpu with slower CPU's? Wondering why we bought 'Royale'????? Steve Willcoxon Shubert Organization
Two things
that come to mind:
update statistics (of course you did, but you didn't say so)layout of tables on disk - check for optimal in each db.
j.
----- Original Message -----
From: "Steve Willcoxon" <stevew@shubertorg.com>
To: <ids@iiug.org>
Sent: Thursday, June 23, 2005 11:55 AM
Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
>
> We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems
> with Unix 5.1B.
>
> Royale: GS1280
> 8 - 1.3 EV7 CPU's
> 8 - gig of memory
>
> Broadhurst: ES80
> 8 - 1.0 EV7 CPU's
> 8 - gig of memory
>
> Majestic: ES45
> 4 - 1.25 EV6 CPU's
> 8 - gig of memory
>
> We have a multi-tier in house application so the database never has more
> than 95 connections.
> We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu).
> The systems are connected with a gigabit switch.
>
> When using different configurations, the resulting throughput speed makes
no
> sense to us.
>
> 1) Royale application server - Majestic db server - fastest combo
> 2) Broadhurst application server - Majestic db server - 2nd fastest
> 3) Royale application server AND db server - much slower than the previous
2
> configurations
> 4) Broadhurst application server - Royale db server - slower than
> configuration #3
>
> At all times the systems are at least 50% idle and usually 75% idle when
> volume is heavy and timing tests were done.
> All timings done with the exact same physical disks (2 spindles in a SAN
we
> configure to the db server).
>
> We would have thought that configuration 3 would be by far the fastest.
> We notice 'delays' when it appears that IDS completely stops for 4 or 5
> seconds with configuration 4, but only 1 second
> with configuration 3. 'Delay' is defined as all our applications stop
(even
> dirty read only) like they are all waiting for the db. There are more
> delays than checkpoints.
>
> It appears to us that Royale runs slower and communicates slower than
> Majestic when it is a db server but is much faster than Majestic as an
> application server.
>
> Does anyone have any ideas why our fastest, newest computer Royale would
run
> slower than the 4 cpu with slower CPU's?
>
>
> Wondering why we bought 'Royale'?????
> Steve Willcoxon
> Shubert Organization
>
>
>
Same
physical disks and hence same database.
The physical disks are dedicated to the database.
Same database means same layout of tables and same update statistics.
This is what is so confusing to us.
Steve Willcoxon
----- Original Message -----
From: "Jack Parker" <vze2qjg5@verizon.net>
To: "Steve Willcoxon" <stevew@shubertorg.com>; <ids@iiug.org>
Sent: Thursday, June 23, 2005 11:40 AM
Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
Two things that come to mind:
update statistics (of course you did, but you didn't say so)layout of tables on disk - check for optimal in each db.
j.
----- Original Message -----
From: "Steve Willcoxon" <stevew@shubertorg.com>
To: <ids@iiug.org>
Sent: Thursday, June 23, 2005 11:55 AM
Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
>
> We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems
> with Unix 5.1B.
>
> Royale: GS1280
> 8 - 1.3 EV7 CPU's
> 8 - gig of memory
>
> Broadhurst: ES80
> 8 - 1.0 EV7 CPU's
> 8 - gig of memory
>
> Majestic: ES45
> 4 - 1.25 EV6 CPU's
> 8 - gig of memory
>
> We have a multi-tier in house application so the database never has more
> than 95 connections.
> We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu).
> The systems are connected with a gigabit switch.
>
> When using different configurations, the resulting throughput speed makes
no
> sense to us.
>
> 1) Royale application server - Majestic db server - fastest combo
> 2) Broadhurst application server - Majestic db server - 2nd fastest
> 3) Royale application server AND db server - much slower than the previous
2
> configurations
> 4) Broadhurst application server - Royale db server - slower than
> configuration #3
>
> At all times the systems are at least 50% idle and usually 75% idle when
> volume is heavy and timing tests were done.
> All timings done with the exact same physical disks (2 spindles in a SAN
we
> configure to the db server).
>
> We would have thought that configuration 3 would be by far the fastest.
> We notice 'delays' when it appears that IDS completely stops for 4 or 5
> seconds with configuration 4, but only 1 second
> with configuration 3. 'Delay' is defined as all our applications stop
(even
> dirty read only) like they are all waiting for the db. There are more
> delays than checkpoints.
>
> It appears to us that Royale runs slower and communicates slower than
> Majestic when it is a db server but is much faster than Majestic as an
> application server.
>
> Does anyone have any ideas why our fastest, newest computer Royale would
run
> slower than the 4 cpu with slower CPU's?
>
>
> Wondering why we bought 'Royale'?????
> Steve Willcoxon
> Shubert Organization
>
>
>
Yes, we use
KAIO.
Same onconfig except number of CPU virtual processors.
We actually ran the Royale with the Majestic onconfig and it was the same
speed as with it own. (only difference 3 or 7 CPU virtual processors)
Steve Willcoxon
----- Original Message -----
From: Clifton M. Bean
To: Steve Willcoxon
Sent: Thursday, June 23, 2005 12:10 PM
Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
Did you turn on KAIO?
Clifton
----- Original Message -----
We pretty much eliminated disk and network because same physical disks on
same network and same problem when application and db on the one system.
We have used the Majestic onconfig on Royale and it made no difference. (see
above to Clifton)
2 spindles?? My complaint also, but the same 2 spindles (4 actually as they
are mirrored) dedicated to the db and raw chunks.
Good cache tho. 99% reads 96% writes
Steve Willcoxon
From: "Jack Parker" <vze2qjg5@verizon.net>
To: "Steve Willcoxon" <stevew@shubertorg.com>; <ids@iiug.org>
Sent: Thursday, June 23, 2005 12:22 PM
Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
I should read more carefully.
All things being equal - which they never are - it has to be at a low
level - related to disk or network.
Try using the Royale as a 4 CPU box instead, set the NUMCPUVPS to 3 instead
of 7. That should make it 'equal' to the Majestic. It would point out a
contention issue if it made things faster.
Speaking of which, only 2 spindles? Contention might very well be your
issue.
Of course you've checked that it's not a memory swapping issue.
j.
----- Original Message -----
From: "Steve Willcoxon" <stevew@shubertorg.com>
To: "Jack Parker" <vze2qjg5@verizon.net>; <ids@iiug.org>
Sent: Thursday, June 23, 2005 1:54 PM
Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
> Same physical disks and hence same database.
> The physical disks are dedicated to the database.
> Same database means same layout of tables and same update statistics.
> This is what is so confusing to us.
>
> Steve Willcoxon
>
> ----- Original Message -----
> From: "Jack Parker" <vze2qjg5@verizon.net>
> To: "Steve Willcoxon" <stevew@shubertorg.com>; <ids@iiug.org>
> Sent: Thursday, June 23, 2005 11:40 AM
> Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
>
>
> Two things that come to mind:
>
> update statistics (of course you did, but you didn't say so)> layout of tables on disk - check for optimal in each db.
>
> j.
> ----- Original Message -----
> From: "Steve Willcoxon" <stevew@shubertorg.com>
> To: <ids@iiug.org>
> Sent: Thursday, June 23, 2005 11:55 AM
> Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233]
>
>
> >
> > We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems
> > with Unix 5.1B.
> >
> > Royale: GS1280
> > 8 - 1.3 EV7 CPU's
> > 8 - gig of memory
> >
> > Broadhurst: ES80
> > 8 - 1.0 EV7 CPU's
> > 8 - gig of memory
> >
> > Majestic: ES45
> > 4 - 1.25 EV6 CPU's
> > 8 - gig of memory
> >
> > We have a multi-tier in house application so the database never has more
> > than 95 connections.
> > We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu).
> > The systems are connected with a gigabit switch.
> >
> > When using different configurations, the resulting throughput speed
makes
> no
> > sense to us.
> >
> > 1) Royale application server - Majestic db server - fastest combo
> > 2) Broadhurst application server - Majestic db server - 2nd fastest
> > 3) Royale application server AND db server - much slower than the
previous
> 2
> > configurations
> > 4) Broadhurst application server - Royale db server - slower than
> > configuration #3
> >
> > At all times the systems are at least 50% idle and usually 75% idle when
> > volume is heavy and timing tests were done.
> > All timings done with the exact same physical disks (2 spindles in a SAN
> we
> > configure to the db server).
> >
> > We would have thought that configuration 3 would be by far the fastest.
> > We notice 'delays' when it appears that IDS completely stops for 4 or 5
> > seconds with configuration 4, but only 1 second
> > with configuration 3. 'Delay' is defined as all our applications stop
> (even
> > dirty read only) like they are all waiting for the db. There are more
> > delays than checkpoints.
> >
> > It appears to us that Royale runs slower and communicates slower than
> > Majestic when it is a db server but is much faster than Majestic as an
> > application server.
> >
> > Does anyone have any ideas why our fastest, newest computer Royale would
> run
> > slower than the 4 cpu with slower CPU's?
> >
> >
> > Wondering why we bought 'Royale'?????
> > Steve Willcoxon
> > Shubert Organization
> >
> >
> >
>
>
>
>
Checkpoints seem to be a little longer on Royale. Read and write cache are the same (read 99% write 96% on both systems). = =20 LRU writes vs disk writes are the same (no fg writes) "hot" disks or dbspaces are the same, if any, since same physical disks = and database. Steve Willcoxon Shubert Organization 800-275-8587 ----- Original Message -----=20 From: Clifton M. Bean=20 To: Steve Willcoxon=20 Sent: Thursday, June 23, 2005 12:39 PM Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower = [5233]=20 How about your checkpoints? Same duration? =20 Read and write cache comparable? =20 LRU writes vs disk writes comparable? Any "hot" disks or dbspaces? Clifton Steve Willcoxon <stevew@shubertorg.com> wrote: Yes, we use KAIO. Same onconfig except number of CPU virtual processors. We actually ran the Royale with the Majestic onconfig and it was the = same speed as with it own. (only difference 3 or 7 CPU virtual = processors) Steve Willcoxon ----- Original Message -----=20 From: Clifton M. Bean=20 To: Steve Willcoxon=20 Sent: Thursday, June 23, 2005 12:10 PM Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower = [5233]=20 Did you turn on KAIO? Clifton Steve Willcoxon <stevew@shubertorg.com> wrote: We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 = systems with Unix 5.1B. Royale: GS1280 8 - 1.3 EV7 CPU's 8 - gig of memory Broadhurst: ES80 8 - 1.0 EV7 CPU's 8 - gig of memory Majestic: ES45 4 - 1.25 EV6 CPU's 8 - gig of memory We have a multi-tier in house application so the database never = has more than 95 connections. We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu). The systems are connected with a gigabit switch. When using different configurations, the resulting throughput = speed makes no sense to us. 1) Royale application server - Majestic db server - fastest = combo 2) Broadhurst application server - Majestic db server - 2nd = fastest 3) Royale application server AND db server - much slower than = the previous 2 configurations 4) Broadhurst application server - Royale db server - slower = than configuration #3 At all times the systems are at least 50% idle and usually 75% = idle when volume is heavy and timing tests were done. All timings done with the exact same physical disks (2 spindles = in a SAN we configure to the db server). We would have thought that configuration 3 would be by far the = fastest. We notice 'delays' when it appears that IDS completely stops for = 4 or 5 seconds with configuration 4, but only 1 second with configuration 3... 'Delay' is defined as all our = applications stop (even dirty read only) like they are all waiting for the db. There are = more delays than checkpoints. It appears to us that Royale runs slower and communicates slower = than Majestic when it is a db server but is much faster than Majestic = as an application server. Does anyone have any ideas why our fastest, newest computer = Royale would run slower than the 4 cpu with slower CPU's? Wondering why we bought 'Royale'????? Steve Willcoxon Shubert Organization
Network configuration? You mention application wait times. Are your NIC's and switches set properly ie 100 full, autonegotiate? I've seen this problem many times where autonegotiation is involved, the NIC is set to 100 full duplex and the switch is set to 100 half duplex or some other bad combination. It could be something as simple as a bad switch, cable or NIC from your description. Do you have any tests scenarios you can run against the database outside of the application that could guage just your database performance? At least you would be able to narrow down your problem. >From: "Steve Willcoxon" <stevew@shubertorg.com> >To: ids@iiug.org >Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233] Date: >Thu, 23 Jun 2005 10:55:02 -0400 (EDT) > >We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems >with Unix 5.1B. > >Royale: GS1280 >8 - 1.3 EV7 CPU's >8 - gig of memory > >Broadhurst: ES80 >8 - 1.0 EV7 CPU's >8 - gig of memory > >Majestic: ES45 >4 - 1.25 EV6 CPU's >8 - gig of memory > >We have a multi-tier in house application so the database never has more >than 95 connections. >We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu). >The systems are connected with a gigabit switch. > >When using different configurations, the resulting throughput speed makes >no >sense to us. > >1) Royale application server - Majestic db server - fastest combo >2) Broadhurst application server - Majestic db server - 2nd fastest >3) Royale application server AND db server - much slower than the previous >2 >configurations >4) Broadhurst application server - Royale db server - slower than >configuration #3 > >At all times the systems are at least 50% idle and usually 75% idle when >volume is heavy and timing tests were done. >All timings done with the exact same physical disks (2 spindles in a SAN we >configure to the db server). > >We would have thought that configuration 3 would be by far the fastest. >We notice 'delays' when it appears that IDS completely stops for 4 or 5 >seconds with configuration 4, but only 1 second >with configuration 3. 'Delay' is defined as all our applications stop >(even >dirty read only) like they are all waiting for the db. There are more >delays than checkpoints. > >It appears to us that Royale runs slower and communicates slower than >Majestic when it is a db server but is much faster than Majestic as an >application server. > >Does anyone have any ideas why our fastest, newest computer Royale would >run >slower than the 4 cpu with slower CPU's? > > >Wondering why we bought 'Royale'????? >Steve Willcoxon >Shubert Organization > >
What is the db doing while the delays are occuring? Can you isolate it to a single query or a single table? Are the network cards configured the same? Is the memory of the same physical speed/confifuration on Royale and Majestic? --- Steve Willcoxon <stevew@shubertorg.com> wrote: > Checkpoints seem to be a little longer on Royale. > Read and write cache are the same (read 99% write > 96% on both systems). = > =20 > LRU writes vs disk writes are the same (no fg > writes) > "hot" disks or dbspaces are the same, if any, since > same physical disks = > and database. > > Steve Willcoxon > Shubert Organization > 800-275-8587 > > ----- Original Message -----=20 > From: Clifton M. Bean=20 > To: Steve Willcoxon=20 > Sent: Thursday, June 23, 2005 12:39 PM > Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster > System Runs Slower = > [5233]=20 > > > How about your checkpoints? Same duration? =20 > Read and write cache comparable? =20 > LRU writes vs disk writes comparable? > Any "hot" disks or dbspaces? > > Clifton > > Steve Willcoxon <stevew@shubertorg.com> wrote: > Yes, we use KAIO. > Same onconfig except number of CPU virtual > processors. > We actually ran the Royale with the Majestic > onconfig and it was the = > same speed as with it own. (only difference 3 or 7 > CPU virtual = > processors) > > Steve Willcoxon > ----- Original Message -----=20 > From: Clifton M. Bean=20 > To: Steve Willcoxon=20 > Sent: Thursday, June 23, 2005 12:10 PM > Subject: Re: Alpha Tru64 IDS 9.40.FC5 - Faster > System Runs Slower = > [5233]=20 > > > Did you turn on KAIO? > Clifton > > Steve Willcoxon <stevew@shubertorg.com> wrote: > > We are running IDS 9.40.FC5 and have the > following 3 Alpha Tru64 = > systems > with Unix 5.1B. > > Royale: GS1280 > 8 - 1.3 EV7 CPU's > 8 - gig of memory > > Broadhurst: ES80 > 8 - 1.0 EV7 CPU's > 8 - gig of memory > > Majestic: ES45 > 4 - 1.25 EV6 CPU's > 8 - gig of memory > > We have a multi-tier in house application so > the database never = > has more > than 95 connections. > We use processor affinity (1-7 for 8 cpu and > 1-3 for 4 cpu). > The systems are connected with a gigabit > switch. > > When using different configurations, the > resulting throughput = > speed makes no > sense to us. > > 1) Royale application server - Majestic db > server - fastest = > combo > 2) Broadhurst application server - Majestic > db server - 2nd = > fastest > 3) Royale application server AND db server - > much slower than = > the previous 2 > configurations > 4) Broadhurst application server - Royale db > server - slower = > than > configuration #3 > > At all times the systems are at least 50% > idle and usually 75% = > idle when > volume is heavy and timing tests were done. > All timings done with the exact same > physical disks (2 spindles = > in a SAN we > configure to the db server). > > We would have thought that configuration 3 > would be by far the = > fastest. > We notice 'delays' when it appears that IDS > completely stops for = > 4 or 5 > seconds with configuration 4, but only 1 > second > with configuration 3... 'Delay' is defined > as all our = > applications stop (even > dirty read only) like they are all waiting > for the db. There are = > more > delays than checkpoints. > > It appears to us that Royale runs slower and > communicates slower = > than > Majestic when it is a db server but is much > faster than Majestic = > as an > application server. > > Does anyone have any ideas why our fastest, > newest computer = > Royale would run > slower than the 4 cpu with slower CPU's? > > > Wondering why we bought 'Royale'????? > Steve Willcoxon > Shubert Organization > > > > >
Everything is either 1000 full, or 100 full (depending on the network), and NO autonegotiation. The systems are connected over a Gigabit network (Cisco 4006 switch). And I believe that it is also 1000 full (1 gig) with NO autonegotiation. Also, configuration #3 has the problem with no network involved. >From my previous response to Jack Parker: We pretty much eliminated disk and network because same physical disks on same network and same problem when application and db on the one system. We have used the Majestic onconfig on Royale and it made no difference. (see above to Clifton) 2 spindles?? My complaint also, but the same 2 spindles (4 actually as they are mirrored) dedicated to the db and raw chunks. Good cache tho. 99% reads 96% writes Steve Willcoxon ----- Original Message ----- From: "Barry Leb" <bjleb@hotmail.com> To: <stevew@shubertorg.com>; <ids@iiug.org> Sent: Thursday, June 23, 2005 2:32 PM Subject: RE: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233] Network configuration? You mention application wait times. Are your NIC's and switches set properly ie 100 full, autonegotiate? I've seen this problem many times where autonegotiation is involved, the NIC is set to 100 full duplex and the switch is set to 100 half duplex or some other bad combination. It could be something as simple as a bad switch, cable or NIC from your description. Do you have any tests scenarios you can run against the database outside of the application that could guage just your database performance? At least you would be able to narrow down your problem. >From: "Steve Willcoxon" <stevew@shubertorg.com> >To: ids@iiug.org >Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233] Date: >Thu, 23 Jun 2005 10:55:02 -0400 (EDT) > >We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems >with Unix 5.1B. > >Royale: GS1280 >8 - 1.3 EV7 CPU's >8 - gig of memory > >Broadhurst: ES80 >8 - 1.0 EV7 CPU's >8 - gig of memory > >Majestic: ES45 >4 - 1.25 EV6 CPU's >8 - gig of memory > >We have a multi-tier in house application so the database never has more >than 95 connections. >We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu). >The systems are connected with a gigabit switch. > >When using different configurations, the resulting throughput speed makes >no >sense to us. > >1) Royale application server - Majestic db server - fastest combo >2) Broadhurst application server - Majestic db server - 2nd fastest >3) Royale application server AND db server - much slower than the previous >2 >configurations >4) Broadhurst application server - Royale db server - slower than >configuration #3 > >At all times the systems are at least 50% idle and usually 75% idle when >volume is heavy and timing tests were done. >All timings done with the exact same physical disks (2 spindles in a SAN we >configure to the db server). > >We would have thought that configuration 3 would be by far the fastest. >We notice 'delays' when it appears that IDS completely stops for 4 or 5 >seconds with configuration 4, but only 1 second >with configuration 3. 'Delay' is defined as all our applications stop >(even >dirty read only) like they are all waiting for the db. There are more >delays than checkpoints. > >It appears to us that Royale runs slower and communicates slower than >Majestic when it is a db server but is much faster than Majestic as an >application server. > >Does anyone have any ideas why our fastest, newest computer Royale would >run >slower than the 4 cpu with slower CPU's? > > >Wondering why we bought 'Royale'????? >Steve Willcoxon >Shubert Organization > >
I would agree. We were told by HP concerning performance problems to 'set' all the NICs and ports in the switches to 1GB and not rely on 'auto negotiate'. It made a noticeable difference. Bob Roussey Spirit Airlines Informix/Unix Administration Phone: 586.741.8991 e-mail: RobertRo@SpiritAir.com -----Original Message----- From: forum.subscriber@iiug.org [mailto:forum.subscriber@iiug.org] On Behalf Of Barry Leb Sent: Thursday, June 23, 2005 2:38 PM To: ids@iiug.org Subject: RE: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5241] Network configuration? You mention application wait times. Are your NIC's and switches set properly ie 100 full, autonegotiate? I've seen this problem many times where autonegotiation is involved, the NIC is set to 100 full duplex and the switch is set to 100 half duplex or some other bad combination. It could be something as simple as a bad switch, cable or NIC from your description. Do you have any tests scenarios you can run against the database outside of the application that could guage just your database performance? At least you would be able to narrow down your problem. >From: "Steve Willcoxon" <stevew@shubertorg.com> >To: ids@iiug.org >Subject: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5233] Date: >Thu, 23 Jun 2005 10:55:02 -0400 (EDT) > >We are running IDS 9.40.FC5 and have the following 3 Alpha Tru64 systems >with Unix 5.1B. > >Royale: GS1280 >8 - 1.3 EV7 CPU's >8 - gig of memory > >Broadhurst: ES80 >8 - 1.0 EV7 CPU's >8 - gig of memory > >Majestic: ES45 >4 - 1.25 EV6 CPU's >8 - gig of memory > >We have a multi-tier in house application so the database never has more >than 95 connections. >We use processor affinity (1-7 for 8 cpu and 1-3 for 4 cpu). >The systems are connected with a gigabit switch. > >When using different configurations, the resulting throughput speed makes >no >sense to us. > >1) Royale application server - Majestic db server - fastest combo >2) Broadhurst application server - Majestic db server - 2nd fastest >3) Royale application server AND db server - much slower than the previous >2 >configurations >4) Broadhurst application server - Royale db server - slower than >configuration #3 > >At all times the systems are at least 50% idle and usually 75% idle when >volume is heavy and timing tests were done. >All timings done with the exact same physical disks (2 spindles in a SAN we >configure to the db server). > >We would have thought that configuration 3 would be by far the fastest. >We notice 'delays' when it appears that IDS completely stops for 4 or 5 >seconds with configuration 4, but only 1 second >with configuration 3. 'Delay' is defined as all our applications stop >(even >dirty read only) like they are all waiting for the db. There are more >delays than checkpoints. > >It appears to us that Royale runs slower and communicates slower than >Majestic when it is a db server but is much faster than Majestic as an >application server. > >Does anyone have any ideas why our fastest, newest computer Royale would >run >slower than the 4 cpu with slower CPU's? > > >Wondering why we bought 'Royale'????? >Steve Willcoxon >Shubert Organization > >
If network configuration is the problem, then one would have to
assume,
that
the third configuration on IDS on royale / application on royale would use
some sort
of loopback through the Network! (instead of the virtual local loopback
device)
You may want to check the connection settings ,
( check resolv.conf, /etc/hosts, /etc/networks , /etc/services,
$INFORMIXSQLHOSTS ..
and compare that with the information from onstat -g ntt..
However, one interesting information would be the WAIT statistics inside
IDS .
Set $ONCONFIG parameter WSTATS to 1 , stop and restart IDS (in case it was
not set before)
then you may get a starting point through the flollowing query:
select sum(cumtime), reason from sysmaster:sysseswts
where cumtime >1000 and reason != 'condition'
group by reason
order by 1 desc;
With kind regards
Tilman
--
Tilman Model-Bosch
IBM Data Management Solutions, Informix Advanced Support
c\\\\o SAP AG
TECHDEV 05
Neurrotstr.16
69190 Walldorf
forum.subscriber@iiug.org wrote on 23/06/2005 22:20:27:
> I would agree. We were told by HP concerning performance problems to
> 'set' all the NICs and ports in the switches to 1GB and not rely on
> 'auto negotiate'. It made a noticeable difference.
>
>
> Bob Roussey
> Spirit Airlines
> Informix/Unix Administration
> Phone: 586.741.8991
> e-mail: RobertRo@SpiritAir.com
Thanks
to all who commented. The problem seems to be the hardware and not
IDS, thank goodness.
To fix the problem, I made the following onconfig changes to reduce sizes
and now Royale is by far the fastest even though we are only using about 4
gig or half of the memory.
PHYSFILE 64000 to 16000
PHYSBUFF 512 to 64
LOGBUFF 512 to 64
BUFFERS 1200000 to 1000000
For those of you using Tru64 this writeup could answer alot of your
questions.
http://www.uni-magdeburg.de/urzs/marvel/vmbug3.html
Steve Willcoxon
The Shubert Organization
----- Original Message -----
From: "Tilman Mode...." <tilman.model-bosch@de.ibm.com>
To: <ids@iiug.org>
Sent: Friday, June 24, 2005 8:29 AM
Subject: RE: Alpha Tru64 IDS 9.40.FC5 - Faster System Runs Slower [5248]
If network configuration is the problem, then one would have to assume,
that
the third configuration on IDS on royale / application on royale would use
some sort
of loopback through the Network! (instead of the virtual local loopback
device)
You may want to check the connection settings ,
( check resolv.conf, /etc/hosts, /etc/networks , /etc/services,
$INFORMIXSQLHOSTS ..
and compare that with the information from onstat -g ntt..
However, one interesting information would be the WAIT statistics inside
IDS .
Set $ONCONFIG parameter WSTATS to 1 , stop and restart IDS (in case it was
not set before)
then you may get a starting point through the flollowing query:
select sum(cumtime), reason from sysmaster:sysseswts
where cumtime >1000 and reason != 'condition'
group by reason
order by 1 desc;
With kind regards
Tilman
--
Tilman Model-Bosch
IBM Data Management Solutions, Informix Advanced Support
c\\\\o SAP AG
TECHDEV 05
Neurrotstr.16
69190 Walldorf
forum.subscriber@iiug.org wrote on 23/06/2005 22:20:27:
> I would agree. We were told by HP concerning performance problems to
> 'set' all the NICs and ports in the switches to 1GB and not rely on
> 'auto negotiate'. It made a noticeable difference.
>
>
> Bob Roussey
> Spirit Airlines
> Informix/Unix Administration
> Phone: 586.741.8991
> e-mail: RobertRo@SpiritAir.com