Slow loading/lots of checkpoints
--0__=GGYxm1203iqFnco9cIc2SoLfFPx3WcXdTnvM0aSSkAu75a2WGDUAUTXI
Content-type: text/plain; charset=us-ascii
Hi all,
I am using IDS Workgroup 7.30 TC3 for Windows NT 4, SP3.
I had unloaded about 5 million rows from a table on one server
and am attempting to load it into an identical table on another server.
The server I am loading the data into is a small IBM Netfinity 3000
350MHz processor with 192M RAM. There is adequate hdd space
but I am having the following problem:
I am loading the data via dbaccess with this command
'load from x1.out insert into x'
where 'x1.out' is the unloaded file and 'x' is the name of the table.
It seems as though data will load for a few seconds and then
everything stops while a checkpoint is performed. My checkpoints
are taking about 25-30 seconds.
I have 7 logical logs that are 25M each.
Is there anyway to setup the onconfig file to work on this meager machine?
ONCONFIG file is attached.
(See attached file: ONCONFIG.ol_ubs_nt_server)
TIA
SMP
--0__=GGYxm1203iqFnco9cIc2SoLfFPx3WcXdTnvM0aSSkAu75a2WGDUAUTXI
Content-type: application/octet-stream;
name="ONCONFIG.ol_ubs_nt_server"
Content-Disposition: attachment; filename="ONCONFIG.ol_ubs_nt_server"
Content-transfer-encoding: base64
#**************************************************************************
#
# INFORMIX SOFTWARE, INC.
#
# Title: onconfig.ol_ubs_nt_server
# Description: Informix Dynamic Server Configuration Parameters
#
#**************************************************************************
# Root Dbspace Configuration
ROOTNAME rootdbs # Root dbspace name
ROOTPATH D:\\IFMXDATA\\ol_ubs_nt_server\\rootdbs_dat.000
# Path for device containing root dbspace
ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes)
ROOTSIZE 30720 # Size of root dbspace (Kbytes)
# Disk Mirroring Configuration Parameters
MIRROR 1 # Mirroring flag (Yes = 1, No = 0)
MIRRORPATH
# Path for device containing mirrored root
MIRROROFFSET 0 # Offset into mirrored device (Kbytes)
# Physical Log Configuration
PHYSDBS ubsspace1 # Location (dbspace) of physical log
PHYSFILE 30000 # Physical log file size (Kbytes)
# Logical Log Configuration
LOGFILES 10 # Number of logical log files
LOGSIZE 500 # Logical log size (Kbytes)
LOG_BACKUP_MODE CONT # Logical log backup mode (MANUAL, CONT)
# Diagnostics
MSGPATH D:\\informix\\ol_ubs_nt_server.log # System message log file path
CONSOLE D:\\informix\\conol_ubs_nt_server.log
# System console message path
ALARMPROGRAM D:\\informix\\etc\\log_full.bat # Alarm program path
# System Diagnostic Script.
# SYSALARMPROGRAM - Full path of the system diagnostic script (e.g.
# c:\\informix\\etc\\evidence.bat.) Set this parameter
# if you want a different Diagnostic Script than
# {INFORMIXDIR}\\etc\\evidence.bat, which is default.
# System Archive Tape Device
TAPEDEV D:\\dbbackup\\db\\dbback # Tape device path
TAPEBLK 16 # Tape block size (Kbytes)
TAPESIZE 250000 # Maximum amount of data to put on tape (Kbytes)
# Log Archive Tape Device
LTAPEDEV D:\\dbbackup\\logs\\logback # Log tape device path
LTAPEBLK 16 # Log tape block size (Kbytes)
LTAPESIZE 50000 # Max amount of data to put on log tape (Kbytes)
# Optical
STAGEBLOB # Informix Dynamic Server/Optical staging area
OPTICAL_LIB_PATH # Location of Optical Subsystem driver DLL
# System Configuration
SERVERNUM 0 # Unique id corresponding to a server instance
DBSERVERNAME ol_ubs_nt_server # Name of default Dynamic Server
DBSERVERALIASES # List of alternate dbservernames
NETTYPE soctcp,1,,NET # Override sqlhosts nettype parameters
DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed env.
RESIDENT 1 # Forced residency flag (Yes = 1, No = 0)
MULTIPROCESSOR 0 # 0 for single-processor, 1 for multi-processor
NUMCPUVPS 2 # Number of user (cpu) vps
SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps to one
NOAGE 0 # Process aging
AFF_SPROC 0 # Affinity start processor
AFF_NPROCS 0 # Affinity number of processors
# Shared Memory Parameters
LOCKS 20000 # Maximum number of locks
BUFFERS 30000 # Maximum number of shared buffers
NUMAIOVPS 1 # Number of IO vps
PHYSBUFF 32 # Physical log buffer size (Kbytes)
LOGBUFF 32 # Logical log buffer size (Kbytes)
LOGSMAX 20 # Maximum number of logical log files
CLEANERS 3 # Number of buffer cleaner processes
SHMBASE 0xc000000 # Shared memory base address
SHMVIRTSIZE 16384 # initial virtual shared memory segment size
SHMADD 4096 # Size of new shared memory segments (Kbytes)
SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited
CKPTINTVL 600 # Check point interval (in sec)
LRUS 4 # Number of LRU queues
LRU_MAX_DIRTY 20 # LRU percent dirty begin cleaning limit
LRU_MIN_DIRTY 5 # LRU percent dirty end cleaning limit
LTXHWM 50
First please do not post MIME, this is a text only newsgroup. Also your
attachement is not easily retrieved.
The too frequent checkpoints (I assume you have CKPTINTVL set to 10-15
minutes {600-900 secs}) are caused by the PHYSICAL log being too small.
Check this by running onstat -l just before the checkpoint and you will
see that the physical log is nearly 75% full which will trigger an early
checkpoint. Increase the size of the physical log by a factor calculated
from the ratio of the actual interval between checkpoints and the desired
interval (presumably CKPTINTVL) divided by 0.75.
Ie your current physical log is 10 MB and CKPTINTVL is 600 but checkpoints
are happening every 50 seconds. You want checkpoints during similar load
to occur 1/12 as often so you will need 12 times the physical logspace.
Right now checkpoint happen at 75% of 10MB or 7.5MB. Thus:
((10 * 0.75) * 12) / 0.75 = 120MB
but I would use the following to be safe and ignore the 75% factor at the
beginning:
(10 * 12) / 0.75 = 160MB
Art S. Kagel
SPasco@unibiz.com wrote:
>
> --0__=GGYxm1203iqFnco9cIc2SoLfFPx3WcXdTnvM0aSSkAu75a2WGDUAUTXI
> Content-type: text/plain; charset=us-ascii
>
> Hi all,
>
> I am using IDS Workgroup 7.30 TC3 for Windows NT 4, SP3.
> I had unloaded about 5 million rows from a table on one server
> and am attempting to load it into an identical table on another server.
>
> The server I am loading the data into is a small IBM Netfinity 3000
> 350MHz processor with 192M RAM. There is adequate hdd space
> but I am having the following problem:
>
> I am loading the data via dbaccess with this command
> 'load from x1.out insert into x'
> where 'x1.out' is the unloaded file and 'x' is the name of the table.
> It seems as though data will load for a few seconds and then
> everything stops while a checkpoint is performed. My checkpoints
> are taking about 25-30 seconds.
>
> I have 7 logical logs that are 25M each.
>
> Is there anyway to setup the onconfig file to work on this meager machine?
> ONCONFIG file is attached.
>
> (See attached file: ONCONFIG.ol_ubs_nt_server)
>
> TIA
> SMP
> --0__=GGYxm1203iqFnco9cIc2SoLfFPx3WcXdTnvM0aSSkAu75a2WGDUAUTXI
> Content-type: application/octet-stream;
> name="ONCONFIG.ol_ubs_nt_server"
> Content-Disposition: attachment; filename="ONCONFIG.ol_ubs_nt_server"
> Content-transfer-encoding: base64
>
> #**************************************************************************
> #
> # INFORMIX SOFTWARE, INC.
> #
> # Title: onconfig.ol_ubs_nt_server
> # Description: Informix Dynamic Server Configuration Parameters
> #
> #**************************************************************************
> # Root Dbspace Configuration
> ROOTNAME rootdbs # Root dbspace name
> ROOTPATH D:\\IFMXDATA\\ol_ubs_nt_server\\rootdbs_dat.000
> # Path for device containing root dbspace
> ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes)
> ROOTSIZE 30720 # Size of root dbspace (Kbytes)
> # Disk Mirroring Configuration Parameters
> MIRROR 1 # Mirroring flag (Yes = 1, No = 0)
> MIRRORPATH
> # Path for device containing mirrored root
> MIRROROFFSET 0 # Offset into mirrored device (Kbytes)
> # Physical Log Configuration
> PHYSDBS ubsspace1 # Location (dbspace) of physical log
> PHYSFILE 30000 # Physical log file size (Kbytes)
> # Logical Log Configuration
> LOGFILES 10 # Number of logical log files
> LOGSIZE 500 # Logical log size (Kbytes)
> LOG_BACKUP_MODE CONT # Logical log backup mode (MANUAL, CONT)
> # Diagnostics
> MSGPATH D:\\informix\\ol_ubs_nt_server.log # System message log file path
> CONSOLE D:\\informix\\conol_ubs_nt_server.log
> # System console message path
> ALARMPROGRAM D:\\informix\\etc\\log_full.bat # Alarm program path
> # System Diagnostic Script.
> # SYSALARMPROGRAM - Full path of the system diagnostic script (e.g.
> # c:\\informix\\etc\\evidence.bat.) Set this parameter
> # if you want a different Diagnostic Script than
> # {INFORMIXDIR}\\etc\\evidence.bat, which is default.
> # System Archive Tape Device
> TAPEDEV D:\\dbbackup\\db\\dbback # Tape device path
> TAPEBLK 16 # Tape block size (Kbytes)
> TAPESIZE 250000 # Maximum amount of data to put on tape (Kbytes)
> # Log Archive Tape Device
> LTAPEDEV D:\\dbbackup\\logs\\logback # Log tape device path
> LTAPEBLK 16 # Log tape block size (Kbytes)
> LTAPESIZE 50000 # Max amount of data to put on log tape (Kbytes)
> # Optical
> STAGEBLOB # Informix Dynamic Server/Optical staging area
> OPTICAL_LIB_PATH # Location of Optical Subsystem driver DLL
> # System Configuration
> SERVERNUM 0 # Unique id corresponding to a server instance
> DBSERVERNAME ol_ubs_nt_server # Name of default Dynamic Server
> DBSERVERALIASES # List of alternate dbservernames
> NETTYPE soctcp,1,,NET # Override sqlhosts nettype parameters
> DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed env.
> RESIDENT 1 # Forced residency flag (Yes = 1, No = 0)
> MULTIPROCESSOR 0 # 0 for single-processor, 1 for multi-processor
> NUMCPUVPS 2 # Number of user (cpu) vps
> SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps to one
> NOAGE 0 # Process aging
> AFF_SPROC 0 # Affinity start processor
> AFF_NPROCS 0 # Affinity number of processors
> # Shared Memory Parameters
> LOCKS 20000 # Maximum number of locks
> BUFFERS 300