[EMFILE][24]: out of shared memory segments, check system SHMSEG

Hi All, I am using IDS 10:00 UC5 on AIX 5.3 and replicating my data on my DR-Site using Enterprise Replication. I am also running an important application on this machine which uses IDS databases. Replication works perfectly fine until I run a daily cutoff process. Whenever this process executes following messages appears in ONLINE.LOG 13:42:54 CDR grouper memory allocation failed (Evaluator replication ID). 13:42:54 shmat: [EMFILE][24]: out of shared memory segments, check system SHMSEG 13:42:54 out of virtual shared memory 13:42:54 shmat: [EMFILE][24]: out of shared memory segments, check system SHMSEG 13:42:54 out of virtual shared memory 13:42:54 CDR grouper memory allocation failed (FanOut update link). 13:42:54 CDR Grouper FanOut thread is aborting. 13:42:54 shmat: [EMFILE][24]: out of shared memory segments, check system SHMSEG 13:42:54 out of virtual shared memory When I defined export EXTSHM=ON in ".profile" of ROOT and INFORMIX users and tried to restart the IDS, I faced following errors therefore i had no other choice than to revert this change. Sun Aug 5 01:15:02 2007 01:15:02 Event alarms enabled. ALARMPROG = '/informix/ids10/etc/alarmprogram.sh' 01:15:02 Booting Language <c> from module <> 01:15:02 Loading Module <CNULL> 01:15:02 Booting Language <builtin> from module <> 01:15:02 Loading Module <BUILTINNULL> 01:15:06 Dynamically allocated new virtual shared memory segment (size 8192KB) 01:15:07 DR: DRAUTO is 0 (Off) 01:15:07 AIX MP latch code enabled 01:15:08 IBM Informix Dynamic Server Version 10.00.UC5 Software Serial Number AAA#B000000 01:15:08 KAIO: error in kaio_READ, kaiocbp = 0x5024c7c8, errno = 14 01:15:08 fildes = 256 (gfd 4), buf = 0x60048c00, nbytes = 4096, offset = 0 01:15:08 Assert Warning: chunk failed sanity check 01:15:08 IBM Informix Dynamic Server Version 10.00.UC5 01:15:08 Who: Session(1, informix@appsec, 0, 50226018) Thread(7, main_loop(), 501fd018, 1) File: rspartn.c Line: 8673 01:15:08 Results: Chunk 2 is being taken OFFLINE. 01:15:08 Action: Restore chunk from archive. 01:15:08 stack trace for pid 20888 written to /informix/ids10/tmp/af.3efde4c 01:15:08 See Also: /informix/ids10/tmp/af.3efde4c 01:15:09 chunk failed sanity check 01:15:10 I/O error, Primary Chunk '/app/db/app_lnk_dbs' -- Offline (sanity) 01:15:11 IBM Informix Dynamic Server Initialized -- Shared Memory Initialized. 01:15:11 Physical Recovery Started at Page (1:31950). 01:15:11 Physical Recovery Complete: 0 Pages Examined, 0 Pages Restored. 01:15:11 Assert Warning: pthdrpage:ptalloc:bad bfget 01:15:11 IBM Informix Dynamic Server Version 10.00.UC5 01:15:11 Who: Session(7, informix@appsec, 0, 50226838) Thread(22, fast_rec, 501fe448, 1) File: rspartn.c Line: 6726 01:15:11 Results: Cannot use TBLSpace page for TBLSpace 2097153 01:15:11 Action: Run 'oncheck -pt 2097153' 01:15:11 stack trace for pid 20888 written to /informix/ids10/tmp/af.3fede4f 01:15:11 See Also: /informix/ids10/tmp/af.3fede4f 01:15:13 pthdrpage:ptalloc:bad bfget 01:15:13 Assert Failed: Chunk 2 is being taken OFFLINE. 01:15:13 IBM Informix Dynamic Server Version 10.00.UC5 01:15:13 Who: Session(7, informix@appsec, 0, 50226838) Thread(22, fast_rec, 501fe448, 1) File: rspartn.c Line: 6744 01:15:13 Results: Cannot Open DBspace 2. 01:15:13 Action: Restore chunk from archive. 01:15:13 stack trace for pid 20888 written to /informix/ids10/tmp/af.3fede4f 01:15:13 See Also: /informix/ids10/tmp/af.3fede4f 01:15:17 Chunk 2 is being taken OFFLINE. 01:15:17 Assert Warning: Chunk 2 is being taken OFFLINE. 01:15:17 IBM Informix Dynamic Server Version 10.00.UC5 01:15:17 Who: Session(7, informix@appsec, 0, 50226838) I will really appreciate, if someone can help me in this regard. Regards, Navaid Arif

Navaid Arif wrote: > Hi All, > I am using IDS 10:00 UC5 on AIX 5.3 and replicating my data on my > DR-Site using Enterprise Replication. I am also running an important > application on this machine which uses IDS databases. Replication works > perfectly fine until I run a daily cutoff process. Whenever this process > executes following messages appears in ONLINE.LOG Need to see the onconfig file, and also onstat -g seg at the time of the problem. > > > 13:42:54 CDR grouper memory allocation failed (Evaluator > replication ID). > 13:42:54 shmat: [EMFILE][24]: out of shared memory segments, > check system SHMSEG > > 13:42:54 out of virtual shared memory > > 13:42:54 shmat: [EMFILE][24]: out of shared memory segments, > check system SHMSEG > > 13:42:54 out of virtual shared memory > > 13:42:54 CDR grouper memory allocation failed (FanOut update link). > 13:42:54 CDR Grouper FanOut thread is aborting. > 13:42:54 shmat: [EMFILE][24]: out of shared memory segments, > check system SHMSEG > > 13:42:54 out of virtual shared memory > > > When I defined export EXTSHM=ON in ".profile" of ROOT and INFORMIX users > and tried to restart the IDS, I faced following errors therefore i had > no other choice than to revert this change. > > Sun Aug 5 01:15:02 2007 > > 01:15:02 Event alarms enabled. ALARMPROG = > '/informix/ids10/etc/alarmprogram.sh' > 01:15:02 Booting Language <c> from module <> > 01:15:02 Loading Module <CNULL> > 01:15:02 Booting Language <builtin> from module <> > 01:15:02 Loading Module <BUILTINNULL> > 01:15:06 Dynamically allocated new virtual shared memory > segment (size 8192KB) > 01:15:07 DR: DRAUTO is 0 (Off) > 01:15:07 AIX MP latch code enabled > 01:15:08 IBM Informix Dynamic Server Version 10.00.UC5 > Software Serial Number AAA#B000000 > 01:15:08 KAIO: error in kaio_READ, kaiocbp = 0x5024c7c8, errno = 14 > 01:15:08 fildes = 256 (gfd 4), buf = 0x60048c00, nbytes = 4096, > offset = 0 > 01:15:08 Assert Warning: chunk failed sanity check > > 01:15:08 IBM Informix Dynamic Server Version 10.00.UC5 > 01:15:08 Who: Session(1, informix@appsec > <mailto:informix@appsec>, 0, 50226018) > Thread(7, main_loop(), 501fd018, 1) > File: rspartn.c Line: 8673 > 01:15:08 Results: Chunk 2 is being taken OFFLINE. > 01:15:08 Action: Restore chunk from archive. > 01:15:08 stack trace for pid 20888 written to > /informix/ids10/tmp/af.3efde4c > 01:15:08 See Also: /informix/ids10/tmp/af.3efde4c > 01:15:09 chunk failed sanity check > > 01:15:10 I/O error, Primary Chunk '/app/db/app_lnk_dbs' -- > Offline (sanity) > 01:15:11 IBM Informix Dynamic Server Initialized -- Shared > Memory Initialized. > > 01:15:11 Physical Recovery Started at Page (1:31950). > 01:15:11 Physical Recovery Complete: 0 Pages Examined, 0 Pages > Restored. > 01:15:11 Assert Warning: pthdrpage:ptalloc:bad bfget > 01:15:11 IBM Informix Dynamic Server Version 10.00.UC5 > 01:15:11 Who: Session(7, informix@appsec > <mailto:informix@appsec>, 0, 50226838) > Thread(22, fast_rec, 501fe448, 1) > File: rspartn.c Line: 6726 > 01:15:11 Results: Cannot use TBLSpace page for TBLSpace 2097153 > 01:15:11 Action: Run 'oncheck -pt 2097153' > 01:15:11 stack trace for pid 20888 written to > /informix/ids10/tmp/af.3fede4f > 01:15:11 See Also: /informix/ids10/tmp/af.3fede4f > 01:15:13 pthdrpage:ptalloc:bad bfget > 01:15:13 Assert Failed: Chunk 2 is being taken OFFLINE. > 01:15:13 IBM Informix Dynamic Server Version 10.00.UC5 > 01:15:13 Who: Session(7, informix@appsec > <mailto:informix@appsec>, 0, 50226838) > Thread(22, fast_rec, 501fe448, 1) > File: rspartn.c Line: 6744 > 01:15:13 Results: Cannot Open DBspace 2. > 01:15:13 Action: Restore chunk from archive. > 01:15:13 stack trace for pid 20888 written to > /informix/ids10/tmp/af.3fede4f > 01:15:13 See Also: /informix/ids10/tmp/af.3fede4f > 01:15:17 Chunk 2 is being taken OFFLINE. > 01:15:17 Assert Warning: Chunk 2 is being taken OFFLINE. > 01:15:17 IBM Informix Dynamic Server Version 10.00.UC5 > 01:15:17 Who: Session(7, informix@appsec > <mailto:informix@appsec>, 0, 50226838) > > I will really appreciate, if someone can help me in this regard. > > > Regards, > *Navaid Arif*