Re: loading a table more efficiently...

I followed all the appropriate advice, including creating 2 more temporary dbspaces of 500Mb each. I've loaded the database minus creating the 12 indexes for the largest table. The process took 3.5 hours. I'm creating the indexes now. I would appreciate your advice on our current system configuration. I haven't taken an RDBMS tuning course yet, but I have been reading some RDBMS literature, the Informix Performance Guide, comments from ths list, etc. I know any performance issue is driven by the performance objective. I'll provide our configuration. Hopefully, you all can highlight certain parameters that can improve the perfornamce of loads/unloads as well as query performance. Hardware: Sun E3000 512MB RAM; one CPU ; no RAID or mirroring internal disk controller; two 4GB internal Sun disks; one external 18GB disk Sun SPARCstation5 64MB RAM, one CPU; no RAID or mirroring two internal sun disks (4 &1 GB) ; one external 9GB sun disk one seperate controller Solaris 2.7 ; IDS 7.30 UC3 Workgroup Edition Informix dbspaces in raw devices Our application is Client/Server using TCP/UDP for IPC (Inter Procees Communications). We are a financial services instiution located in Belize. Our application is primarily OLTP but we do have requirements for large DSS queries periodically. The application uses one connection to Informix and several processes handle requests from the win9x clients where the GUI front end is installed. #************************************************************************** # # INFORMIX SOFTWARE, INC. # # Title: onconfig.bbank # Description: Informix Dynamic Server Configuration Parameters # #************************************************************************** # Root Dbspace Configuration ROOTNAME rootdbs # Root dbspace name ROOTPATH /dev/rpltdbs # Path for device containing root dbspace ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes) ROOTSIZE 50000 # Size of root dbspace (Kbytes) # Disk Mirroring Configuration Parameters MIRROR 0 # Mirroring flag (Yes = 1, No = 0) MIRRORPATH # Path for device containing mirrored root MIRROROFFSET 0 # Offset into mirrored device (Kbytes) # Physical Log Configuration PHYSDBS rootdbs # Location (dbspace) of physical log PHYSFILE 1000 # Physical log file size (Kbytes) # Logical Log Configuration LOGFILES 94 # Number of logical log files LOGSIZE 500 # Logical log size (Kbytes) # Diagnostics MSGPATH /sfw/informix/online.log # System message log file path CONSOLE /dev/console # System console message path ALARMPROGRAM /sfw/informix/etc/log_full.sh # Alarm program pathSYSALARMPROGRAM /sfw/informix/etc/evidence.sh # System Alarm program path TBLSPACE_STATS 1 # System Archive Tape Device #TAPEDEV /dev/null # Tape device path TAPEDEV /work/backups/informix/ontape # Tape device path TAPEBLK 16 # Tape block size (Kbytes) TAPESIZE 3000000 # Maximum amount of data to put on tape (Kbytes) # Log Archive Tape Device LTAPEDEV /dev/null # Log tape device path LTAPEBLK 16 # Log tape block size (Kbytes) LTAPESIZE 3000000 # Max amount of data to put on log tape (Kbytes) # Optical STAGEBLOB # Informix Dynamic Server/Optical staging area # System Configuration SERVERNUM 0 # Unique id corresponding to a DynamicServer instance DBSERVERNAME BB_IDS_MAIN_shm # Name of default database server DBSERVERALIASES BB_IDS_MAIN # List of alternate dbservernames NETTYPE tlitcp,1,50,CPU # Network conection type description NETTYPE ipcshm,1,50,NET # Network conection type description DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed env. RESIDENT 0 # Forced residency flag (Yes = 1, No = 0) MULTIPROCESSOR 0 # 0 for single-processor, 1 formulti-processor NUMCPUVPS 1 # Number of user (cpu) vps SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps toone NOAGE 0 # Process aging AFF_SPROC 0 # Affinity start processor AFF_NPROCS 0 # Affinity number of processors # Shared Memory Parameters LOCKS 100000 # Maximum number of locks BUFFERS 64000 # Maximum number of shared buffers NUMAIOVPS 2 # Number of IO vps PHYSBUFF 32 # Physical log buffer size (Kbytes) LOGBUFF 32 # Logical log buffer size (Kbytes)LOGSMAX 100 # Maximum number of logical log files CLEANERS 2 # Number of buffer cleaner processes SHMBASE 0xa000000 # Shared memory base address SHMVIRTSIZE 128000 # initial virtual shared memory segment size SHMADD 8192 # Size of new shared memory segments (Kbytes) SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited CKPTINTVL 900 # Check point interval (in sec) LRUS 8 # Number of LRU queues LRU_MAX_DIRTY 5 # LRU percent dirty begin cleaning limit LRU_MIN_DIRTY 2 # LRU percent dirty end cleaning limit LTXHWM 50 # Long transaction high water markpercentage LTXEHWM 60 # Long transaction high water mark (exclusive) TXTIMEOUT 0x12c # Transaction timeout (in sec) STACKSIZE 32 # Stack size (Kbytes) # System Page Size # BUFFSIZE - Dynamic Server no longer supports this configuration parameter. # To determine the page size used by Dynamic Server on your platform # see the last line of output from the command, 'onstat -b'. # Recovery Variables # OFF_RECVRY_THREADS: # Number of parallel worker threads during fast recovery or an offline restore. # ON_RECVRY_THREADS: # Number of parallel worker threads during an online restore. OFF_RECVRY_THREADS 10 # Default number of offline workerthreads ON_RECVRY_THREADS 1 # Default number of online worker threads # Data Replication Variables # DRAUTO: 0 manual, 1 retain type, 2 reverse type DRAUTO 0 # DR automatic switchover DRINTERVAL 30 # DR max time between DR buffer flushes (in sec) DRTIMEOUT 30 # DR network timeout (in sec)DRLOSTFOUND /sfw/informix/etc/dr.lostfound # DR lost+found file path # CDR Variables CDR_LOGBUFFERS 2048 # size of log reading buffer pool (Kbytes) CDR_EVALTHREADS 1,2 # evaluator threads (per-cpu-vp,additional) CDR_DSLOCKWAIT 5 # DS lockwait timeout (seconds) CDR_QUEUEMEM 4096 # Maximum amount of memory for any CDR queue (Kbytes)@@NL

Denmark B. Weatherburn wrote in message <95v28l$mh0$1@news.xmission.com>... > >I followed all the appropriate advice, including creating 2 more temporary >dbspaces of 500Mb each. I've loaded the database minus creating the 12 >indexes for the largest table. The process took 3.5 hours. I'm creating the >indexes now. Be careful about creating temp dbspaces of different sizes. The big problem is, the engine will allocate new temp tables to each one in turn, and you really cannot predict which one will be used. So that implies, for general use, all temp dbspaces should be the same size - so they basically run out together! Perhaps the engine can concentrate on spaces with more remaining space, but I haven't read about that. Someone else may comment. You can direct specific processes to use specific temp dbspaces (ie huuuuuge or reserved ones) by exporting the shell variable DBSPACETEMP prior to running the process. We've used this to direct some large end of month processes to specific temp spaces so they don't fight with other work. I've heard some people say that causing the engine to use external files for sorting during index builds is quicker, but that's flatly contradicted by the performance tuning manuals. Perhaps it depends on the configuration and the hardware, and perhaps those people had UNIX file systems on separate devices vs. their temp spaces on the same disk as other dbspaces. It's definitely worth testing this for your own machinery. Check out the PSORT_DBTEMP shell variable in the manuals. And related ones.