Performance problem with simple query
AIX 4.2.1
Online 7.3 UD6
This weekend we converted our database from version 5 to version 7.3. We
have noticed the following problem trying to execute the following select :
The estimated cost is to high. (The select takes 26 secs) We have run UPDATE
STATISTICS !!!
Number of rows in artikel : 29000
Number of rows in artomschr : 87000
------
select artikel.artikelnr , artikel.artnr from artomschr, artikel
where artikel.artikelnr matches "42*"
and artikel.artnr = artomschr.artnr
and artomschr.taal = "NL"
Estimated Cost: 9401
Estimated # of Rows Returned: 4371
1) marcv.artikel: INDEX PATH
(1) Index Keys: artikelnr (desc)
Lower Index Filter: marcv.artikel.artikelnr MATCHES '42*'
2) marcv.artomschr: INDEX PATH
(1) Index Keys: artnr taal (Key-Only)
DYNAMIC HASH JOIN (Build Outer)
Dynamic Hash Filters: marcv.artikel.artnr = marcv.artomschr.artnr
Other Join Filters: marcv.artomschr.taal = 'NL'
CREATE TABLE artikel
( artnr SERIAL not null,
artikelnr CHAR(20) not null,
firmastuk CHAR(1) not null,
rotatiekode CHAR(1) )
EXTENT SIZE 16000 NEXT SIZE 2000
LOCK MODE ROW ;
CREATE UNIQUE INDEX i_artikel01 ON artikel(artikelnr DESC) ;
CREATE UNIQUE INDEX i_artikel02 ON artikel(artnr) ;
CREATE UNIQUE INDEX i_artikel03 ON artikel(artikelnr) ;
CREATE UNIQUE INDEX i_artikel04 ON artikel(firmastuk,artikelnr) ;
CREATE TABLE artomschr
( artnr INTEGER not null,
taal CHAR(2) not null,
omschr CHAR(40) not null )
EXTENT SIZE 6400 NEXT SIZE 640
LOCK MODE ROW ;
CREATE INDEX i_amaai01 ON artomschr(artnr) ;
CREATE UNIQUE INDEX i_artomschr01 ON artomschr(artnr,taal) ;
#**************************************************************************
#
# INFORMIX SOFTWARE, INC.
#
# Title: onconfig.std
# Description: Informix Dynamic Server Configuration Parameters
#
#**************************************************************************
# Root Dbspace Configuration
ROOTNAME rootdbs # Root dbspace name
ROOTPATH /dev/kroots # Path for device containing root dbspace
ROOTOFFSET 0 # Offset of root dbspace into device
(Kbytes)
ROOTSIZE 307200 # Size of root dbspace (Kbytes)
# Disk Mirroring Configuration Parameters
MIRROR 1 # Mirroring flag (Yes = 1, No = 0)
MIRRORPATH # Path for device containing mirrored root
MIRROROFFSET 0 # Offset into mirrored device (Kbytes)
# Physical Log Configuration
PHYSDBS rootdbs # Location (dbspace) of physical log
PHYSFILE 35000 # Physical log file size (Kbytes)
# Logical Log Configuration
LOGFILES 70 # Number of logical log files
LOGSIZE 1500 # Logical log size (Kbytes)
# Diagnostics
MSGPATH /usr/informixv7/online.kmc # System message log file path
CONSOLE /dev/console # System console message path
ALARMPROGRAM /usr/informixv7/etc/log_full.sh # Alarm program pathSYSALARMPROGRAM /usr/informixv7/etc/evidence.sh # System Alarm program path
TBLSPACE_STATS 1
# System Archive Tape Device
TAPEDEV /dev/rmt0 # Tape device path
TAPEBLK 16 # Tape block size (Kbytes)
TAPESIZE 4500000 # Maximum amount of data to put on tape
(Kbytes)
# Log Archive Tape Device
LTAPEDEV /dev/rmt0 # Log tape device path
LTAPEBLK 16 # Log tape block size (Kbytes)
LTAPESIZE 2500000 # Max amount of data to put on log tape
(Kbytes)
# Optical
STAGEBLOB # Informix Dynamic Server/Optical staging
area
# System Configuration
SERVERNUM 204 # Unique id corresponding to a DynamicServer instance
DBSERVERNAME onl7kmc # Name of default database server
DBSERVERALIASES onl7kmcs # List of alternate dbservernames
NETTYPE soctcp,1,10,NET # Configure poll thread(s) for nettype
NETTYPE ipcshm,1,,CPU # Configure poll thread(s) for nettype
DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed
env.
RESIDENT 0 # Forced residency flag (Yes = 1, No = 0)
MULTIPROCESSOR 0 # 0 for single-processor, 1 formulti-processor
NUMCPUVPS 1 # Number of user (cpu) vps
SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps toone
NOAGE 0 # Process aging
AFF_SPROC 0 # Affinity start processor
AFF_NPROCS 0 # Affinity number of processors
# Shared Memory Parameters
LOCKS 30000 # Maximum number of locks
BUFFERS 2000 # Maximum number of shared buffers
NUMAIOVPS 8 # Number of IO vps
PHYSBUFF 32 # Physical log buffer size (Kbytes)
LOGBUFF 32 # Logical log buffer size (Kbytes)LOGSMAX 125 # Maximum number of logical log files
CLEANERS 8 # Number of buffer cleaner processes
SHMBASE 0x30000000 # Shared memory base address
SHMVIRTSIZE 8000 # initial virtual shared memory segment size
SHMADD 8192 # Size of new shared memory segments
(Kbytes)
SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited
CKPTINTVL 600 # Check point interval (in sec)
LRUS 8 # Number of LRU queues
LRU_MAX_DIRTY 12 # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY 6 # LRU percent dirty end cleaning limit
LTXHWM 40 # Long transaction high water markpercentage
LTXEHWM 50 # Long transaction high water mark
(exclusive)
TXTIMEOUT 0x12c # Transaction timeout (in sec)
STACKSIZE 32 # Stack size (Kbytes)
# System Page Size
# BUFFSIZE - Dynamic Server no longer supports this configuration parameter.
# To determine the page size used by Dynamic Server on your
platform
# see the last line of output from the command, 'onstat -b'.
# Recovery Variables
# OFF_RECVRY_THREADS:
# Number of parallel worker threads during fast recovery or an offline
restore.
# ON_RECVRY_THREADS:
# Number of parallel worker threads during an online restore.
OFF_RECVRY_THREADS 10 # Default number of offline workerthreads
ON_RECVRY_THREADS 1 # Default number of online worker threads
# Data Replication Variables
# DRAUTO: 0 manual, 1 retain type, 2 reverse type
DRAUTO 0 # DR automatic switchover
DRINTERVAL 30 # DR max time between DR buffer flushes (in
sec)
DRTIMEOUT 30 # DR network timeout (in sec)DRLOSTFOUND /usr/informixv7/etc/dr.lostfound # DR lost+found file path
# CDR Variables
CDR_LOGBUFFERS 2048
Willaert Doris wrote:
>
> AIX 4.2.1
> Online 7.3 UD6
>
> This weekend we converted our database from version 5 to version 7.3. We
> have noticed the following problem trying to execute the following select :
> The estimated cost is to high. (The select takes 26 secs) We have run UPDATE
> STATISTICS !!!
>
> Number of rows in artikel : 29000
> Number of rows in artomschr : 87000
>
> ------
> select artikel.artikelnr , artikel.artnr from artomschr, artikel
> where artikel.artikelnr matches "42*"
> and artikel.artnr = artomschr.artnr
> and artomschr.taal = "NL">
> Estimated Cost: 9401
> Estimated # of Rows Returned: 4371
>
> 1) marcv.artikel: INDEX PATH
>
> (1) Index Keys: artikelnr (desc)
> Lower Index Filter: marcv.artikel.artikelnr MATCHES '42*'
>
> 2) marcv.artomschr: INDEX PATH
>
> (1) Index Keys: artnr taal (Key-Only)
>
> DYNAMIC HASH JOIN (Build Outer)
> Dynamic Hash Filters: marcv.artikel.artnr = marcv.artomschr.artnr
>
> Other Join Filters: marcv.artomschr.taal = 'NL'
There can be several reasons for an initial installation of 7.xx
replacing 5.xx being slower. For example, yes you ran UPDATE
STATISTICS but did you do so the same way you did in 5.xx? IDS 7.xx
has new UPDATE STATISTICS features that you MUST use properly to get
best performance without running more stats than you need. (Get my
dostats.ec utility that takes care of this for you, it is part of the
package utils2_ak in the IIUG Software Repository.)
Another thing is that the HASH table IDS is created needs temp disk
space. In 5.xx there were no Dynamic Hash Joins but similar temp space
was allocated on $TMPDIR or $DBTEMP. Now that goes into the DBSPACES
named in $DBSPACETEMP or $PSORT_DBTEMP for sort-work files (same as 5.x)
or if these are not set usually ROOTDBS is used for these temp tables.
Your ROOTDBS may be over burdened. Create temp dbspaces and add them
to $DBSPACETEMP if you have not done so.
In your ONCONFIG you have only one CPU VP declared and set
MULTIPROCESSOR to 0 but SINGLE_CPU_VP is also 0 which will allow you
to add CPU VPs on the fly but also turns off several internal engine
optimizations that avoid many latches and other inter-VP coordination
that may not be needed. If you really only need or want one CPU VP
then set SINGLE_CPU_VP to 1.
Art S. Kagel
I have came accoss this problem when moving form online 5 to IDS 7.3 too.
It was soveved by addinf teh venviroment variable:
NO_SUBQ=1;
Hans.
Willaert Doris wrote:
> AIX 4.2.1
> Online 7.3 UD6
>
> This weekend we converted our database from version 5 to version 7.3. We
> have noticed the following problem trying to execute the following select :
> The estimated cost is to high. (The select takes 26 secs) We have run UPDATE
> STATISTICS !!!
>
> Number of rows in artikel : 29000
> Number of rows in artomschr : 87000
>
> ------
> select artikel.artikelnr , artikel.artnr from artomschr, artikel
> where artikel.artikelnr matches "42*"
> and artikel.artnr = artomschr.artnr
> and artomschr.taal = "NL">
> Estimated Cost: 9401
> Estimated # of Rows Returned: 4371
>
> 1) marcv.artikel: INDEX PATH
>
> (1) Index Keys: artikelnr (desc)
> Lower Index Filter: marcv.artikel.artikelnr MATCHES '42*'
>
> 2) marcv.artomschr: INDEX PATH
>
> (1) Index Keys: artnr taal (Key-Only)
>
> DYNAMIC HASH JOIN (Build Outer)
> Dynamic Hash Filters: marcv.artikel.artnr = marcv.artomschr.artnr
>
> Other Join Filters: marcv.artomschr.taal = 'NL'
>
> CREATE TABLE artikel
> ( artnr SERIAL not null,
> artikelnr CHAR(20) not null,
> firmastuk CHAR(1) not null,
> rotatiekode CHAR(1) )
> EXTENT SIZE 16000 NEXT SIZE 2000
> LOCK MODE ROW ;
> CREATE UNIQUE INDEX i_artikel01 ON artikel(artikelnr DESC) ;
> CREATE UNIQUE INDEX i_artikel02 ON artikel(artnr) ;
> CREATE UNIQUE INDEX i_artikel03 ON artikel(artikelnr) ;
> CREATE UNIQUE INDEX i_artikel04 ON artikel(firmastuk,artikelnr) ;>
> CREATE TABLE artomschr
> ( artnr INTEGER not null,
> taal CHAR(2) not null,
> omschr CHAR(40) not null )
> EXTENT SIZE 6400 NEXT SIZE 640
> LOCK MODE ROW ;
> CREATE INDEX i_amaai01 ON artomschr(artnr) ;
> CREATE UNIQUE INDEX i_artomschr01 ON artomschr(artnr,taal) ;>
> #**************************************************************************
> #
> # INFORMIX SOFTWARE, INC.
> #
> # Title: onconfig.std
> # Description: Informix Dynamic Server Configuration Parameters
> #
> #**************************************************************************
>
> # Root Dbspace Configuration
>
> ROOTNAME rootdbs # Root dbspace name
> ROOTPATH /dev/kroots # Path for device containing root dbspace
> ROOTOFFSET 0 # Offset of root dbspace into device
> (Kbytes)
> ROOTSIZE 307200 # Size of root dbspace (Kbytes)>
> # Disk Mirroring Configuration Parameters
>
> MIRROR 1 # Mirroring flag (Yes = 1, No = 0)
> MIRRORPATH # Path for device containing mirrored root
> MIRROROFFSET 0 # Offset into mirrored device (Kbytes)>
> # Physical Log Configuration
>
> PHYSDBS rootdbs # Location (dbspace) of physical log
> PHYSFILE 35000 # Physical log file size (Kbytes)>
> # Logical Log Configuration
>
> LOGFILES 70 # Number of logical log files
> LOGSIZE 1500 # Logical log size (Kbytes)>
> # Diagnostics
>
> MSGPATH /usr/informixv7/online.kmc # System message log file path
> CONSOLE /dev/console # System console message path
> ALARMPROGRAM /usr/informixv7/etc/log_full.sh # Alarm program path> SYSALARMPROGRAM /usr/informixv7/etc/evidence.sh # System Alarm program path
> TBLSPACE_STATS 1>
> # System Archive Tape Device
>
> TAPEDEV /dev/rmt0 # Tape device path
> TAPEBLK 16 # Tape block size (Kbytes)
> TAPESIZE 4500000 # Maximum amount of data to put on tape
> (Kbytes)>
> # Log Archive Tape Device
>
> LTAPEDEV /dev/rmt0 # Log tape device path
> LTAPEBLK 16 # Log tape block size (Kbytes)
> LTAPESIZE 2500000 # Max amount of data to put on log tape
> (Kbytes)>
> # Optical
>
> STAGEBLOB # Informix Dynamic Server/Optical staging
> area
>
> # System Configuration
>
> SERVERNUM 204 # Unique id corresponding to a Dynamic> Server instance
> DBSERVERNAME onl7kmc # Name of default database server
> DBSERVERALIASES onl7kmcs # List of alternate dbservernames
> NETTYPE soctcp,1,10,NET # Configure poll thread(s) for nettype
> NETTYPE ipcshm,1,,CPU # Configure poll thread(s) for nettype
> DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed
> env.
> RESIDENT 0 # Forced residency flag (Yes = 1, No = 0)
>
> MULTIPROCESSOR 0 # 0 for single-processor, 1 for> multi-processor
> NUMCPUVPS 1 # Number of user (cpu) vps
> SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps to> one
>
> NOAGE 0 # Process aging
> AFF_SPROC 0 # Affinity start processor
> AFF_NPROCS 0 # Affinity number of processors>
> # Shared Memory Parameters
>
> LOCKS 30000 # Maximum number of locks
> BUFFERS 2000 # Maximum number of shared buffers
> NUMAIOVPS 8 # Number of IO vps
> PHYSBUFF 32 # Physical log buffer size (Kbytes)
> LOGBUFF 32 # Logical log buffer size (Kbytes)> LOGSMAX 125 # Maximum number of logical log files
> CLEANERS 8 # Number of buffer cleaner processes
> SHMBASE 0x30000000 # Shared memory base address
> SHMVIRTSIZE 8000 # initial virtual shared memory segment size
> SHMADD 8192 # Size of new shared memory segments
> (Kbytes)
> SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited
> CKPTINTVL 600 # Check point interval (in sec)
> LRUS 8 # Number of LRU queues
> LRU_MAX_DIRTY 12 # LRU percent dirty begin cleaning limit
> LRU_MIN_DIRTY 6 # LRU percent dirty end cleaning limit
> LTXHWM 40 # Long transaction high water mark> percentage
> LTXEHWM 50 # Long transaction high water mark
> (exclusive)
> TXTIMEOUT 0x12c # Transaction timeout (in sec)
> STACKSIZE 32 # Stack size (Kbytes)>
> # System Page Size
> # BUFFSIZE - Dynamic Server no longer supports this configuration parameter.
> # To determine the page size used by Dynamic Server on your
> platform
> # see the last line of output from the command, 'onstat -b'.
>
> # Recovery Variables
> # OFF_RECVRY_THREADS:
> # Number of parallel worker threads during fast recovery or an offline
> restore.
> # ON_RECVRY_THREADS:
> # Number of parallel worker threads during an online restore.
>
> OFF_RECVRY_THREADS 10 # Default number of off
In article <374B86BC.1C6705FF@tusc.com.au>,
Hans <hans@tusc.com.au> wrote:
> I have came accoss this problem when moving form online 5 to IDS 7.3
too.
>
> It was soveved by addinf teh venviroment variable:
>
> NO_SUBQ=1;
>
> Hans.
>
> Willaert Doris wrote:
>
> > AIX 4.2.1
> > Online 7.3 UD6
> >
> > This weekend we converted our database from version 5 to version
7.3. We
> > have noticed the following problem trying to execute the following
select :
> > The estimated cost is to high. (The select takes 26 secs) We have
run UPDATE
> > STATISTICS !!!
> >
> > Number of rows in artikel : 29000
> > Number of rows in artomschr : 87000
> >
> > ------
> > select artikel.artikelnr , artikel.artnr from artomschr, artikel
> > where artikel.artikelnr matches "42*"
> > and artikel.artnr = artomschr.artnr
> > and artomschr.taal = "NL"> >
> > Estimated Cost: 9401
> > Estimated # of Rows Returned: 4371
> >
> > 1) marcv.artikel: INDEX PATH
> >
> > (1) Index Keys: artikelnr (desc)
> > Lower Index Filter: marcv.artikel.artikelnr MATCHES '42*'
> >
> > 2) marcv.artomschr: INDEX PATH
> >
> > (1) Index Keys: artnr taal (Key-Only)
> >
> > DYNAMIC HASH JOIN (Build Outer)
> > Dynamic Hash Filters: marcv.artikel.artnr =
marcv.artomschr.artnr
> >
> > Other Join Filters: marcv.artomschr.taal = 'NL'
> >
> > CREATE TABLE artikel
> > ( artnr SERIAL not null,
> > artikelnr CHAR(20) not null,
> > firmastuk CHAR(1) not null,
> > rotatiekode CHAR(1) )
> > EXTENT SIZE 16000 NEXT SIZE 2000
> > LOCK MODE ROW ;
> > CREATE UNIQUE INDEX i_artikel01 ON artikel(artikelnr DESC) ;
> > CREATE UNIQUE INDEX i_artikel02 ON artikel(artnr) ;
> > CREATE UNIQUE INDEX i_artikel03 ON artikel(artikelnr) ;
> > CREATE UNIQUE INDEX i_artikel04 ON artikel(firmastuk,artikelnr) ;> >
> > CREATE TABLE artomschr
> > ( artnr INTEGER not null,
> > taal CHAR(2) not null,
> > omschr CHAR(40) not null )
> > EXTENT SIZE 6400 NEXT SIZE 640
> > LOCK MODE ROW ;
> > CREATE INDEX i_amaai01 ON artomschr(artnr) ;
> > CREATE UNIQUE INDEX i_artomschr01 ON artomschr(artnr,taal) ;> >
> >
[ ... snipped ... see original posting for $ONCONFIG file ]
My understanding is:
This parameter mentioned by Hans is NO_SUBQF = 1
(I did check this via strings command)
You will need this only to avoid subquery flattening, if
this causes your sluggish behavior.
In the query of the original posting I can not see any
subquery beeing executed
--
Richard Kofler
debis Systemhaus EDVg
Vienna / Austria
--== Sent via Deja.com http://www.deja.com/ ==--
---Share what you know. Learn what you don't.---
Hans wrote: > > I have came accoss this problem when moving form online 5 to IDS 7.3 too. > > It was soveved by addinf teh venviroment variable: > > NO_SUBQ=1; s/b NO_SUBQF > Hans, this only disables correlated sub-query rewriting and the example query does not have subqueries. Also if you do have this problem a better long term solution is to add the missing join indexes causing the sub-query flattening to slow things down. The subq flattening only slows queries if the indexes needed to perform the join contained in the restructured query are not there and a table scan or hash-join are attempted instead. Adding the indexes make the restructured sub-query MUCH faster than using NO_SUBQF. Anyway thanks for helping Willaert out, keep contributing. Art S. Kagel
Related threads
- onbar -c -F in Windows Informix instance
- Anyone... SQLCODE=-668, ISAM error=-1
- Not using the 100% logical log page size alloacted to informix