Re: Too many open files - Isam error 104
In additon: The solaris host is configured to hold 65536 open files per
process.
>From sysdef-command:
*
* Process Resource Limit Tunables (Current:Maximum)
*
0x0000000000000100:0x0000000000010000 file descriptors
*
I understand that this value is the absolute kernel-limit for a shell
with all child-processes. We did a pfiles command on all
oninit-processes and count more than 63000 open file descriptores. They
can also be found in /proc/oninit-PID/fd.
Most of the file descriptors describe (as I assume) tcp connections:
21: S_IFCHR mode:0000 dev:305,0 ino:52962 uid:0 gid:0 rdev:146,718
O_RDWR|O_NDELAY
/devices/pseudo/tcp6@0:tcp6
22: S_IFCHR mode:0000 dev:305,0 ino:21689 uid:0 gid:0 rdev:146,2440
O_RDWR|O_NDELAY
/devices/pseudo/tcp6@0:tcp6
Most of the oninit processes (about 30) hold more than 3000 of those
descriptors. Do they count when reaching the limit of open files? It's
not very likely because user informix starts the server with an ulimit
of 256 files and this limit is by far exceeded. But who knows...?
From: informix-list-bounces@iiug.org
[mailto:informix-list-bounces@iiug.org] On Behalf Of Habichtsberg,
Reinhard
Sent: Wednesday, May 11, 2011 1:04 PM
To: informix-list@iiug.org
Subject: Too many open files - Isam error 104
Hi all,
we have an Informix-4ge program that stops with error 104 "too many open
files" while trying to open a violation table. Same program has run for
years without this error. The code has not been changed.
The schema of two tables has changed:
stop violations table for xxx_neu;
drop table xxx_neu_vio;
drop table xxx_neu_dia;
alter table xxx_neu add (verarb_status SMALLINT);
start violations table for xxx_neu;
set indexes xxx_n_1 filtering;
alter table yyy_neu add (anmelde_grund CHAR(2),
abmelde_grund CHAR(2),
svs67 CHAR(2));
Those columns are not in use of the program yet.
Parts of the program runs on new application server (Solaris 10 Zone).
May be the pogram runs faster than before.
The Informix Server is 11.50FC7W3. OS is Solaris 10.
We checked some limits on the appservers and the database server host.
Ulimit is set to 256 for the appusers and for user informix before
starting the databaseserver.
We couldn't see many open files on the appserver. The informix
databaseserver has more than 3000 concurrent sessions, most of them
ontlitcp connections.
What would you suggest?
TIA,
Reinhard.
Hello Reinhard,
hmm not much anyway:
-->>Most of the oninit processes (about 30) hold more than 3000 of
those
-->>descriptors. Do they count when reaching the limit of open files?
It's
-->>not very likely because user informix starts the server with an
ulimit
-->>of 256 files and this limit is by far exceeded. But who knows...?
oninit is setuid root so it can increase it if i recall correctly.
oninit procs share filedesc's afaik since threads can switch between
vps.
i would try setting sqlidebug
eq
export SQLIDEBUG=2:/tmp/somefile
run the program and use sqliprint to dump the info.
also you could try and use truss
eq
truss ./yourprogram > somefile 2>&1
maybe you find a hint which may case this.
What does support say about this; i assume the repro is pretty easy??
See you
Superboer
On 11 mei, 15:04, "Habichtsberg, Reinhard" <RHabichtsb...@arz-
emmendingen.de> wrote:
> In additon: The solaris host is configured to hold 65536 open files per
> process.
>
> >From sysdef-command:
>
> *
>
> * Process Resource Limit Tunables (Current:Maximum)
>
> *
>
> 0x0000000000000100:0x0000000000010000 file descriptors
>
> *
>
> I understand that this value is the absolute kernel-limit for a shell
> with all child-processes. We did a pfiles command on all
> oninit-processes and count more than 63000 open file descriptores. They
> can also be found in /proc/oninit-PID/fd.
>
> Most of the file descriptors describe (as I assume) tcp connections:
>
> 21: S_IFCHR mode:0000 dev:305,0 ino:52962 uid:0 gid:0 rdev:146,718
>
> O_RDWR|O_NDELAY
>
> /devices/pseudo/tcp6@0:tcp6
>
> 22: S_IFCHR mode:0000 dev:305,0 ino:21689 uid:0 gid:0 rdev:146,2440
>
> O_RDWR|O_NDELAY
>
> /devices/pseudo/tcp6@0:tcp6
>
> Most of the oninit processes (about 30) hold more than 3000 of those
> descriptors. Do they count when reaching the limit of open files? It's
> not very likely because user informix starts the server with an ulimit
> of 256 files and this limit is by far exceeded. But who knows...?
>
> From: informix-list-boun...@iiug.org
> [mailto:informix-list-boun...@iiug.org] On Behalf Of Habichtsberg,
> Reinhard
> Sent: Wednesday, May 11, 2011 1:04 PM
> To: informix-l...@iiug.org
> Subject: Too many open files - Isam error 104
>
> Hi all,
>
> we have an Informix-4ge program that stops with error 104 "too many open
> files" while trying to open a violation table. Same program has run for
> years without this error. The code has not been changed.
>
> The schema of two tables has changed:
>
> stop violations table for xxx_neu;
>
> drop table xxx_neu_vio;>
> drop table xxx_neu_dia;>
> alter table xxx_neu add (verarb_status SMALLINT);>
> start violations table for xxx_neu;
>
> set indexes xxx_n_1 filtering;
>
> alter table yyy_neu add (anmelde_grund CHAR(2),>
> abmelde_grund CHAR(2),
>
> svs67 CHAR(2));
>
> Those columns are not in use of the program yet.
>
> Parts of the program runs on new application server (Solaris 10 Zone).
> May be the pogram runs faster than before.
>
> The Informix Server is 11.50FC7W3. OS is Solaris 10.
>
> We checked some limits on the appservers and the database server host.
> Ulimit is set to 256 for the appusers and for user informix before
> starting the databaseserver.
>
> We couldn't see many open files on the appserver. The informix
> databaseserver has more than 3000 concurrent sessions, most of them
> ontlitcp connections.
>
> What would you suggest?
>
> TIA,
>
> Reinhard.
The kernel max_files per process or for the system overall may be exceeded.
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.
On Thu, May 12, 2011 at 11:19 AM, Superboer <superboer7@t-online.de> wrote:
> Hello Reinhard,
>
>
> hmm not much anyway:
>
> -->>Most of the oninit processes (about 30) hold more than 3000 of
> those
> -->>descriptors. Do they count when reaching the limit of open files?
> It's
> -->>not very likely because user informix starts the server with an
> ulimit
> -->>of 256 files and this limit is by far exceeded. But who knows...?
>
> oninit is setuid root so it can increase it if i recall correctly.
> oninit procs share filedesc's afaik since threads can switch between
> vps.
>
> i would try setting sqlidebug
>
> eq
>
> export SQLIDEBUG=2:/tmp/somefile
> run the program and use sqliprint to dump the info.
>
> also you could try and use truss
> eq
>
> truss ./yourprogram > somefile 2>&1
>
> maybe you find a hint which may case this.
>
> What does support say about this; i assume the repro is pretty easy??
>
>
> See you
>
> Superboer
>
>
>
>
> On 11 mei, 15:04, "Habichtsberg, Reinhard" <RHabichtsb...@arz-
> emmendingen.de> wrote:
> > In additon: The solaris host is configured to hold 65536 open files per
> > process.
> >
> > >From sysdef-command:
> >
> > *
> >
> > * Process Resource Limit Tunables (Current:Maximum)
> >
> > *
> >
> > 0x0000000000000100:0x0000000000010000 file descriptors
> >
> > *
> >
> > I understand that this value is the absolute kernel-limit for a shell
> > with all child-processes. We did a pfiles command on all
> > oninit-processes and count more than 63000 open file descriptores. They
> > can also be found in /proc/oninit-PID/fd.
> >
> > Most of the file descriptors describe (as I assume) tcp connections:
> >
> > 21: S_IFCHR mode:0000 dev:305,0 ino:52962 uid:0 gid:0 rdev:146,718
> >
> > O_RDWR|O_NDELAY
> >
> > /devices/pseudo/tcp6@0:tcp6
> >
> > 22: S_IFCHR mode:0000 dev:305,0 ino:21689 uid:0 gid:0 rdev:146,2440
> >
> > O_RDWR|O_NDELAY
> >
> > /devices/pseudo/tcp6@0:tcp6
> >
> > Most of the oninit processes (about 30) hold more than 3000 of those
> > descriptors. Do they count when reaching the limit of open files? It's
> > not very likely because user informix starts the server with an ulimit
> > of 256 files and this limit is by far exceeded. But who knows...?
> >
> > From: informix-list-boun...@iiug.org
> > [mailto:informix-list-boun...@iiug.org] On Behalf Of Habichtsberg,
> > Reinhard
> > Sent: Wednesday, May 11, 2011 1:04 PM
> > To: informix-l...@iiug.org
> > Subject: Too many open files - Isam error 104
> >
> > Hi all,
> >
> > we have an Informix-4ge program that stops with error 104 "too many open
> > files" while trying to open a violation table. Same program has run for
> > years without this error. The code has not been changed.
> >
> > The schema of two tables has changed:
> >
> > stop violations table for xxx_neu;
> >
> > drop table xxx_neu_vio;> >
> > drop table xxx_neu_dia;> >
> > alter table xxx_neu add (verarb_status SMALLINT);> >
> > start violations table for xxx_neu;
> >
> > set indexes xxx_n_1 filtering;
> >
> > alter table yyy_neu add (anmelde_grund CHAR(2),> >
> > abmelde_grund CHAR(2),
> >
> > svs67 CHAR(2));
> >
> > Those columns are not in use of the program yet.
> >
> > Parts of the program runs on new application server (Solaris 10 Zone).
> > May be the pogram runs faster than before.
> >
> > The Informix Server is 11.50FC7W3. OS is Solaris 10.
> >
> > We checked some limits on the appservers and the database server host.
> > Ulimit is set to 256 for the appusers and for user informix before
> > starting the databaseserver.
> >
> > We couldn't see many open files on the appserver. The informix
> > databaseserver has more than 3000 concurrent sessions, most of them
> > ontlitcp connections.
> >
> > What would you suggest?
> >
> > TIA,
> >
> > Reinhard.
>
> _______________________________________________
> Informix-list mailing list
> Informix-list@iiug.org
> http://www.iiug.org/mailman/listinfo/informix-list
>