-25588 error after CPU & fiber channel failure

HPUX B.11.23 on HP BL870c running IDS 10.00 FC5. This is the replacement server for our current product rp7410 server. So, it's not in production yet. We had a CPU fail and one of our 4 fiber channel paths fail to the SAN that holds the chunks for our IDS database and ever since getting these fixed I'm getting the following error when run one of our 3rd-party apps. Technically the FC didn't fail but somehow when the CPU was swapped one of the FC paths changed. HP can't explain why but c13t0d1 became c15t0d1 and the /etc/lvmtab needed updating to reflect the change. The appl process cannot connect to the database server zoom. See error number -25588. Database cars not found or not correct format Before these failures the app I'm getting the error on did work. I have done the following: - checked onconf, sqlhosts to make sure they didn't somehow change - bounced the IDS engine - rebooted the server - checked dbaccess & isql and other Informix apps to see if can get to the database - all work fine. - I checked all kernel parameters to make sure nothing got changed somehow when the CPU was swapped (checked against the production server). All look ok. I checked this as the error does have a note about possible shared memory kernel parameter mis-configurations. - a bit drastic but I did a oninit -ivy and redid the dbimports of the databases. Dbaccess/isql works but the app still doesn't. I wasted about two days trying to resolve this, what am I missing? John

More information on our setup. If anyone has an idea of what to look at or try let me know. Otherwise the only thing left I haven't tried is whipping the server and start fresh. kernel parameters semaem Static 16384 16384 16384 - semmni Static 16384 16384 2048 0.3% semmns Static 32767 32767 4096 4.6% semmnu Static 2000 2000 256 - semmsl Dynamic 2048 2048 2048 - semume Static 100 100 100 - semvmx Static 32767 32767 32767 - shmmax Dynamic 1073741824 1073741824 1073741824 50.0 shmmni Dynamic 400 400 400 3.2% shmseg Dynamic 300 300 300 - Environmental variables INFORMIXDIR=/opt/informix ONCONFIG=onconf.cars INFORMIXSERVER=zoom our onconf file #************************************************************************** # # Licensed Material - Property Of IBM # # "Restricted Materials of IBM" # # IBM Informix Dynamic Server # (c) Copyright IBM Corporation 1996, 2005 All rights reserved. # # Title: onconfig.std # Description: IBM Informix Dynamic Server Configuration Parameters # #************************************************************************** # Root Dbspace Configuration ROOTNAME root # Root dbspace name ROOTPATH /opt/informix/dev/root.1 # Path for device containing root dbspace ROOTOFFSET 0 # Offset of root dbspace into device (Kbytes) ROOTSIZE 2048000 # Size of root dbspace (Kbytes) # Disk Mirroring Configuration Parameters MIRROR 0 # Mirroring flag (Yes = 1, No = 0) MIRRORPATH # Path for device containing mirrored root MIRROROFFSET 0 # Offset into mirrored device (Kbytes) # Physical Log Configuration PHYSDBS physlog_dbs # Location (dbspace) of physical log PHYSFILE 135138 # Physical log file size (Kbytes)#PHYSFILE 1459214 # used root and this number for oninit -ivy # Logical Log Configuration LOGFILES 50 # Number of logical log files LOGSIZE 8192 # Logical log size (Kbytes)LOG_BACKUP_MODE MANUAL # Logical log backup mode (MANUAL, CONT) # Security # DBCREATE_PERMISSION: # By default any user can create a database. Uncomment DBCREATE_PERMISSON to # limit database creation to a specific user. Add a new DBCREATE_PERMISSION # line for each permitted user. DBCREATE_PERMISSION informix DBCREATE_PERMISSION root # IFX_EXTEND_ROLE: # 0 => Disable use of EXTEND role to control who can register # external routines. This is the default behaviour. # 1 => Enable use of EXTEND role to control who can register # external routines. IFX_EXTEND_ROLE 1 # To control the usage of EXTEND role. # Tablespace Tablespace Configuration in Root Dbspace TBLTBLFIRST 0 # First extent size (Kbytes) (0 = default) TBLTBLNEXT 0 # Next extent size (Kbytes) (0 = default) # Diagnostics MSGPATH /opt/informix/Logs/cars.log # System message log file path CONSOLE /dev/console # System console message path # To automatically backup logical logs, edit alarmprogram.sh and set # BACKUPLOGS=Y ALARMPROGRAM /opt/informix/etc/log_full.sh # Alarm program path ALRM_ALL_EVENTS 0 # Triggers ALARMPROGRAM for any event occur TBLSPACE_STATS 1 # Maintain tblspace statistics # System Archive Tape Device TAPEDEV /dev/null #rmt/c16t0d1BEST # Tape device path TAPEBLK 32 # Tape block size (Kbytes) TAPESIZE 0 # Maximum amount of data to put on tape (Kbytes) # Log Archive Tape Device LTAPEDEV /opt/informix/tlog/current_trans_log # Log tape device path LTAPEBLK 32 # Log tape block size (Kbytes) LTAPESIZE 0 # Max amount of data to put on log tape (Kbytes) # Optical STAGEBLOB # Informix Dynamic Server staging area # System Configuration SERVERNUM 0 # Unique id corresponding to a OnLine instance DBSERVERNAME zoom # Name of default database server DBSERVERALIASES carsitcp # List of alternate dbservernames NETTYPE ipcshm,3,400,CPU # Configure poll thread(s) for nettype NETTYPE soctcp,5,100,NET # Configure poll thread(s) for nettype DEADLOCK_TIMEOUT 60 # Max time to wait of lock in distributed env. # added 07/01/08 DEF_TABLE_LOCKMODE ROW # MULTIPROCESSOR 1 # 0 for single-processor, 1 for multi-processor SINGLE_CPU_VP 0 # If non-zero, limit number of cpu vps to one # Shared Memory Parameters BUFFERPOOL size=2K,buffers=200000,lrus=127,lru_min_dirty=2.000000,lru_max_dirty=4.000000 LOCKS 700000 # Maximum number of locks PHYSBUFF 32 # Physical log buffer size (Kbytes) LOGBUFF 32 # Logical log buffer size (Kbytes) CLEANERS 127 # Number of buffer cleaner processes SHMBASE 0x0L # Shared memory base address SHMVIRTSIZE 524304 # initial virtual shared memory segment size SHMADD 32768 # Size of new shared memory segments (Kbytes) SHMTOTAL 0 # Total shared memory (Kbytes). 0=>unlimited CKPTINTVL 900 # Check point interval (in sec) TXTIMEOUT 0x12c # Transaction timeout (in sec) STACKSIZE 128 # Stack size (Kbytes) # Dynamic Logging # DYNAMIC_LOGS: # 2 : server automatically add a new logical log when necessary. (ON) # 1 : notify DBA to add new logical logs when necessary. (ON) # 0 : cannot add logical log on the fly. (OFF) # # When dynamic logging is on, we can have higher values for LTXHWM/LTXEHWM, # because the server can add new logical logs during long transaction rollback. # However, to limit the number of new logical logs being added, LTXHWM/LTXEHWM # can be set to smaller values. # # If dynamic logging is off, LTXHWM/LTXEHWM need to be set to smaller values # to avoid long transaction rollback hanging the server due to lack of logical # log space, i.e. 50/60 or lower. # # In case of system configured with CDR, the difference between LTXHWM and # LTXEHWM should be atleast 30% so that we could minimize log overrun issue. DYNAMIC_LOGS 0 LTXHWM 50 LTXEHWM 60 # System Page Size # BUFFSIZE - OnLine no longer supports this configuration parameter. # To determine the page size used by OnLine on your platform # see the last line of output from the command, 'onstat -b'. # Recovery Variables # OFF_RECVRY_THREADS: # Number of parallel worker threads during fast recovery or an offline restore. # ON_RECVRY_THREADS: # Number of parallel worker threads during an online restore. OFF_RECVRY_THREADS 10 # Default number of offline worker threads ON_RECVRY_THREADS 1 # Default number of online worker threads # Data Replication Variables # DRAUTO: 0 manual, 1 retain type, 2 reverse type DRAUTO 0 # DR automatic switchover DRINTERVAL 30 # DR max time between DR buffer flushes (in sec) DRTIMEOUT 30 # DR network timeout (in sec) DRLOSTFOUND /opt/informix/etc/dr.lostfound # DR lost+found file path DRIDXAUTO 0 # DR automatic index repair. 0=off, 1=on # CDR Variables CDR_EVALTHREADS 1,2 # evaluator threads (per-cpu-vp,additional) CDR_DSLOCKWAIT 5 # DS lockwait timeout (seconds) CDR_QUEUEMEM 4096 # Maximum amount of memory for any CDR queue (Kbytes) CDR_NIFCOMPRESS 0 # Link level compression (-1 never, 0 none, 9 max) CDR_SERIAL 0,0 # Serial Column Sequence CDR_DBSPACE # dbspace for syscdr database CDR_QHDR_DBSPACE # CDR queue dbspace (default same as catalog) CDR_QDATA_SBSPACE # List of CDR queue smart blob spaces@@NL@

John, See if the app can connect if it uses the network connection (carsitcp) instead of the shared memory connection. Mike