long checkpoint

This message is in MIME format. Since your mail reader does not understand this format, some or all of this message may not be legible. ------_=_NextPart_000_01C007B8.97F708A0 Content-Type: text/plain; charset="iso-8859-1" I have just recently migrated from IDS 7.31.FC4XE to IDS 2000 9.21.FC1 (Two days ago). The system has just lived through a 6+ minute checkpoint. There appears to be no I/O issues, other than I/O just stopped. A peculiarity I have notice is this: Occasionally I see one oninit process (tracked back to number 11cpu vp) running at 98-99 percent of a real cpu for 4 to 5 minutes. This vp is the one with NO POLL THREAD. Is it possible this is blocking the checkpoint. I'm contemplating placing a soc poll thread on this cpu vp. - 10 CPU-vps - 8 soc poll threads on 8 cpu-vps - 1 stream poll thread on 1 cpu-vp - 1 on free cpu vp ( no poll threads) Detailed info follows: System info: - IDS 2000 9.21.FC1 - HP V2500 (12 cpu's) - HPUX 11.00 - 8 Gig memory onconfig and various onstat output <<onstat.txt>> regards Jim Namenek phone: 330-490-5929 fax: 330-490-5735 email: namenej@diebold.com ------_=_NextPart_000_01C007B8.97F708A0 Content-Type: text/plain; name="onstat.txt" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="onstat.txt" Informix Dynamic Server 2000 Version 9.21.FC1 -- On-Line -- Up = 20:08:27 -- 4237312 Kbytes Configuration File: /mnt/informix/bold6_baanp/etc/onconfig.bold6_baanp #***********************************************************************= *** # # INFORMIX SOFTWARE, INC. # # Title: onconfig.std # Description: Informix Dynamic Server 2000 Configuration Parameters # #***********************************************************************= *** # Root Dbspace Configuration ROOTNAME rootdbs # Root dbspace nameROOTPATH /dev/infx.p/rootdbs # Path for device containing root = dbspace ROOTOFFSET 0 # Offset of root dbspace into device = (Kbytes) ROOTSIZE 300000 # Size of root dbspace (Kbytes) # Disk Mirroring Configuration Parameters MIRROR 0 # Mirroring flag (Yes =3D 1, No =3D 0) MIRRORPATH # Path for device containing mirrored =root MIRROROFFSET 0 # Offset into mirrored device (Kbytes) # Physical Log Configuration PHYSDBS rootdbs # Location (dbspace) of physical log PHYSFILE 200000 # Physical log file size (Kbytes) # Logical Log Configuration LOGFILES 424 # Number of logical log files LOGSIZE 12000 # Logical log size (Kbytes) # Diagnostics=20 MSGPATH /mnt/informix/online.bold6_baanp.log=20 # System message log file path CONSOLE /dev/console # System console message path ALARMPROGRAM /mnt/informix/bold6_baanp/etc/no_log.sh # Alarm program =path TBLSPACE_STATS 1 # Maintain tblspace statistics # System Archive Tape Device TAPEDEV /dev/rmt/1m # Tape device path=09 TAPEBLK 20000 # Tape block size (Kbytes) TAPESIZE 100000000 # Maximum amount of data to put on tape = (Kbytes) # Log Archive Tape Device LTAPEDEV /dev/rmt/2m # Log tape device path LTAPEBLK 16 # Log tape block size (Kbytes) LTAPESIZE 23000000 # Max amount of data to put on log tape = (Kbytes) # Optical STAGEBLOB # Informix Dynamic Server 2000 staging = area=20 # System Configuration SERVERNUM 3 # Unique id corresponding to a OnLine =instance DBSERVERNAME bold6_baanp_ipc # Name of default database server DBSERVERALIASES bold6_baanp_fddi1,bold6_baanp_fddi2=20 # List of alternate dbservernames NETTYPE soctcp,8,60,CPU # Configure poll thread(s) for nettype NETTYPE ipcstr,1,100,CPU # Configure poll thread(s) for nettype NETTYPE sqlmux,,, # Configure poll thread(s) for nettype DEADLOCK_TIMEOUT 60 # Max time to wait of lock in =distributed env. RESIDENT 1 # Forced residency flag (Yes =3D 1, No = =3D 0) MULTIPROCESSOR 1 # 0 for single-processor, 1 for =multi-processor NUMCPUVPS 10 # Number of user (cpu) vps SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps =to one NOAGE 1 # Process aging AFF_SPROC 0 # Affinity start processor AFF_NPROCS 0 # Affinity number of processors # Shared Memory Parameters LOCKS 1500000 # Maximum number of locks BUFFERS 300000 # Maximum number of shared buffers NUMAIOVPS 2 # Number of IO vps PHYSBUFF 128 # Physical log buffer size (Kbytes) LOGBUFF 64 # Logical log buffer size (Kbytes)LOGSMAX 480 # Maximum number of logical log files CLEANERS 128 # Number of buffer cleaner processes SHMBASE 0x0 # Shared memory base address SHMVIRTSIZE 3407872 # initial virtual shared memory segment =size SHMADD 1048576 # Size of new shared memory segments = (Kbytes) SHMTOTAL 0 # Total shared memory (Kbytes). = 0=3D>unlimited CKPTINTVL 300 # Check point interval (in sec) LRUS 128 # Number of LRU queues LRU_MAX_DIRTY 2 # LRU percent dirty begin cleaning =limit LRU_MIN_DIRTY 0 # LRU percent dirty end cleaning limit LTXHWM 50 # Long transaction high water mark =percentage LTXEHWM 60 # Long transaction high water mark = (exclusive) TXTIMEOUT 0x12c # Transaction timeout (in sec) STACKSIZE 256 # Stack size (Kbytes) # System Page Size # BUFFSIZE - OnLine no longer supports this configuration parameter. # To determine the page size used by OnLine on your platform # see the last line of output from the command, 'onstat -b'. # Recovery Variables # OFF_RECVRY_THREADS: # Number of parallel worker threads during fast recovery or an offline = restore. # ON_RECVRY_THREADS: # Number of parallel worker threads during an online restore. OFF_RECVRY_THREADS 25 # Default number of offline worker =threads ON_RECVRY_THREADS 1 # Default number of online worker =threads # Data Replication Variables DRINTERVAL 30 # DR max time between DR buffer flushes = (in sec) DRTIMEOUT 30 # DR network timeout (in sec)DRLOSTFOUND /mnt/informix/dr.lostfound # DR lost+found file path # CDR Variables CDR_EVALTHREADS 1,2 # evaluator threads = (per-cpu-vp,additional) CDR_DSLOCKWAIT 5 # DS lockwait timeout

First, please do not post MIME/HTTP, this is a text newsgroup. Thanks. See below for notes. "Namenek, James H." wrote: > > I have just recently migrated from IDS 7.31.FC4XE to IDS 2000 9.21.FC1 (Two days ago). The system has just lived through a 6+ minute checkpoint. There appears to be no I/O issues, other than I/O just stopped. > > A peculiarity I have notice is this: > Occasionally I see one oninit process (tracked back to number 11cpu vp) running at 98-99 percent of a real cpu for 4 to 5 minutes. This vp is the one with NO POLL THREAD. Is it possible this is blocking the checkpoint. I'm contemplating placing a soc poll thread on this cpu vp. > - 10 CPU-vps > - 8 soc poll threads on 8 cpu-vps > - 1 stream poll thread on 1 cpu-vp > - 1 on free cpu vp ( no poll threads) > > Detailed info follows: > System info: > - IDS 2000 9.21.FC1 > - HP V2500 (12 cpu's) > - HPUX 11.00 > - 8 Gig memory > > onconfig and various onstat output > > <<onstat.txt>> > > regards > Jim Namenek > > phone: 330-490-5929 > fax: 330-490-5735 > email: namenej@diebold.com > > Informix Dynamic Server 2000 Version 9.21.FC1 -- On-Line -- Up = > 20:08:27 -- 4237312 Kbytes > > Configuration File: /mnt/informix/bold6_baanp/etc/onconfig.bold6_baanp > #***********************************************************************= > *** > # > # INFORMIX SOFTWARE, INC. > # > # Title: onconfig.std > # Description: Informix Dynamic Server 2000 Configuration Parameters > # > #***********************************************************************= > *** > > # Root Dbspace Configuration > > ROOTNAME rootdbs # Root dbspace name> ROOTPATH /dev/infx.p/rootdbs # Path for device containing root = > dbspace > ROOTOFFSET 0 # Offset of root dbspace into device = > (Kbytes) > ROOTSIZE 300000 # Size of root dbspace (Kbytes)> > # Disk Mirroring Configuration Parameters > > MIRROR 0 # Mirroring flag (Yes =3D 1, No =3D 0) > MIRRORPATH # Path for device containing mirrored => root > MIRROROFFSET 0 # Offset into mirrored device (Kbytes)> > # Physical Log Configuration > > PHYSDBS rootdbs # Location (dbspace) of physical log > PHYSFILE 200000 # Physical log file size (Kbytes) Physical log should be in a separate dbspace from ROOTNAME preferably on a separate spindle(s). Same for logical logs. > > # Logical Log Configuration > > LOGFILES 424 # Number of logical log files > LOGSIZE 12000 # Logical log size (Kbytes)> > # Diagnostics=20 > > MSGPATH /mnt/informix/online.bold6_baanp.log=20 > # System message log file path > CONSOLE /dev/console # System console message path > ALARMPROGRAM /mnt/informix/bold6_baanp/etc/no_log.sh # Alarm program => path > TBLSPACE_STATS 1 # Maintain tblspace statistics Turn TBLSPACE_STATS off, it is very expensive in production loads. > # System Archive Tape Device > > TAPEDEV /dev/rmt/1m # Tape device path=09 > TAPEBLK 20000 # Tape block size (Kbytes) Are you sure your tape drives and the HPUX drivers are actually writing 20MB blocks? Most systems max out at about 1MB. > TAPESIZE 100000000 # Maximum amount of data to put on tape = > (Kbytes)> > # Log Archive Tape Device > > LTAPEDEV /dev/rmt/2m # Log tape device path > LTAPEBLK 16 # Log tape block size (Kbytes) > LTAPESIZE 23000000 # Max amount of data to put on log tape = > (Kbytes)> > # Optical > > STAGEBLOB # Informix Dynamic Server 2000 staging = > area=20 > > # System Configuration > > SERVERNUM 3 # Unique id corresponding to a OnLine => instance > DBSERVERNAME bold6_baanp_ipc # Name of default database server > DBSERVERALIASES bold6_baanp_fddi1,bold6_baanp_fddi2=20 > # List of alternate dbservernames > NETTYPE soctcp,8,60,CPU # Configure poll thread(s) for nettype > NETTYPE ipcstr,1,100,CPU # Configure poll thread(s) forNEVER NEVER put TCP or STR listeners in CPU VPs it burns CPU cycles polling and is not as responsive as using NET VPs for these. Use CPU VPs for shared memory (SHM) listeners ONLY and exclusively use CPU VPS for SHM listeners). I know the manual disagrees, the manual is wrong here as in several specifics. nettype > NETTYPE sqlmux,,, # Configure poll thread(s) for nettype > DEADLOCK_TIMEOUT 60 # Max time to wait of lock in => distributed env. > RESIDENT 1 # Forced residency flag (Yes =3D 1, NoOn HP You may be better with RESIDENT -1 which will allow the engine to combine the resident and first virtual segment into one. = > =3D 0) > > MULTIPROCESSOR 1 # 0 for single-processor, 1 for => multi-processor > NUMCPUVPS 10 # Number of user (cpu) vps > SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps => to one > > NOAGE 1 # Process aging > AFF_SPROC 0 # Affinity start processor > AFF_NPROCS 0 # Affinity number of processors> > # Shared Memory Parameters > > LOCKS 1500000 # Maximum number of locks > BUFFERS 300000 # Maximum number of shared buffers Your buffer turnover rate is once every 150 secs or 23 times an hour. This number should be single digits per hour. Looks like you need more buffers which is only going to make your checkpoints longer but will improve performance. You also show almost 152,000 sequential scans over 20 hours or more than 7500 per hour that's more than 2 per second! You need to review your indexes and queries and make sure stats are updated as per recommendations in the Performance Guide. > NUMAIOVPS 2 # Number of IO vps Are you using KAIO? If so try NUMAIOVPS 4 to 6 if not NUMAIOVPS should be a few more than the number of chunks. > PHYSBUFF 128 # Physical log buffer size (Kbytes) > LOGBUFF 64 # Logical log buffer size (Kbytes)> LOGSMAX 480 # Maximum number of logical log files > CLEANERS 128 # Number of buffer cleaner processes > SHMBASE 0x0 # Shared memory base address > SHMVIRTSIZE 3407872 # initial virtual shared memory segment => size > SHMADD 1048576 # Size of new shared memory segments = > (Kbytes) > SHMTOTAL 0 # Total shared memory (Kbytes). = > 0=3D>unlimited > CKPTINTVL 300 # Check point interval (in sec) > LRUS 128 # Number of LRU queues > LRU_MAX_DIRTY 2 # LRU percent dirty begin cleaning =@@NL