network errors
On HP-UX 11.11 with IDS 9.40FC3, users suddenly couldn't connect and the message log filled with listener-thread errors -25573 (oserr 233, "no buffer space available") and a few -25580 (oserr 232, "connection reset by peer"). Art Kagel suggested network/poll-thread saturation and advised more poll threads or extra ports with DBSERVERALIASes, but the poster's connection counts were low (~250 max, errors recurring at only 30 connections) and bouncing the instance didn't help. Fernando Nunes noted such HP-UX 232/233 errors are often OS-side (memory/performance) and that current S&S still gives limited 9.4 support, so IBM should be called. The poster contacted IBM, but no outcome or fix was ever posted despite follow-up requests.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
OS: HP/UX 11.11 IDS: 9.4FC3 (we are in the process of upgrading, but new server/db not ready for production yet) We started seeing these errors in our log Friday when users complained of not being able to connect. We checked back over time and there have been a few here and there, but Friday we had 50+ and today we have 100+ and counting and are users are having a difficult time connecting/using the system. The majority are: listener-thread: err = -25573: oserr = 233: errstr = : Network driver cannot accept a connection on the port. but we also see a few: listener-thread: err = -25580: oserr = 232: errstr = : System error occurred in network function. Nothing has changed on the server recently and network folks are telling us that nothing has changed with them. Any ideas on what/where to look for/check/change to stop these errors? TIA, Randy
On HP/UX errno 232 is "Connection reset by peer" and 233 id "No buffer space available". The latter, more common error, sounds like the network drivers are being saturated. How many active concurrent connections do you have at the times when this is happening? A single poll threadr can handle up to about 1000 connections, however, IBM recommends configuring so that a thread handles no more than about 350 connections. One solution may be to increase the number of poll threads, however, it if is new connections that are the problem, then you probably should configure one or two additional ports (each with its own DBSERVERALIAS) so that there will be multiple listener threads listening for new connections and set things up so that the users are balanced across the two or three ports. You will have to bounce the server to implement this solution. BTW, in IDS 11.70 you can increase the number of listener threads on a single port in the ONCONFIG file or dynamically making configuring and resolving problems like this easier. IDS 11.70 also has additional features to reduce the overhead of connecting to the server which reduced the load so these problems are less common. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) IIUG Board of Directors (art@iiug.org) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Mon, Dec 13, 2010 at 3:44 PM, Kennedy, Randy <RKennedy@scottsdaleaz.gov>wrote: > OS: HP/UX 11.11 IDS: 9.4FC3 (we are in the process of upgrading, > but new server/db not ready for production yet) > > We started seeing these errors in our log Friday when users complained > of not being able to connect. We checked back over time and there have > been a few here and there, but Friday we had 50+ and today we have 100+ > and counting and are users are having a difficult time connecting/using > the system. > > The majority are: > listener-thread: err = -25573: oserr = 233: errstr = : Network driver > cannot accept a connection on the port. > > but we also see a few: > listener-thread: err = -25580: oserr = 232: errstr = : System error > occurred in network function. > > Nothing has changed on the server recently and network folks are telling > us that nothing has changed with them. > Any ideas on what/where to look for/check/change to stop these errors? > > TIA, > Randy > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --000e0cd25deeba53080497511690
We only have 1 port and the highest maximum connections count from the
log is 268 earlier this year and has been consistently between 210-250
for the past couple of years.
Our onconfig is set as:
NETTYPE soctcp,4,150,NET
NETTYPE ipcshm,1,25,NET
Thanks,
Randy
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
Art Kagel
Sent: Monday, December 13, 2010 2:07 PM
To: ids@iiug.org
Subject: Re: network errors [22209]
On HP/UX errno 232 is "Connection reset by peer" and 233 id "No buffer
space
available". The latter, more common error, sounds like the network
drivers
are being saturated. How many active concurrent connections do you have
at
the times when this is happening? A single poll threadr can handle up to
about 1000 connections, however, IBM recommends configuring so that a
thread
handles no more than about 350 connections. One solution may be to
increase
the number of poll threads, however, it if is new connections that are
the
problem, then you probably should configure one or two additional ports
(each with its own DBSERVERALIAS) so that there will be multiple
listener
threads listening for new connections and set things up so that the
users
are balanced across the two or three ports. You will have to bounce the
server to implement this solution.
BTW, in IDS 11.70 you can increase the number of listener threads on a
single port in the ONCONFIG file or dynamically making configuring and
resolving problems like this easier. IDS 11.70 also has additional
features
to reduce the overhead of connecting to the server which reduced the
load so
these problems are less common.
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art@iiug.org)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other
organization with which I am associated either explicitly, implicitly,
or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.
On Mon, Dec 13, 2010 at 3:44 PM, Kennedy, Randy
<RKennedy@scottsdaleaz.gov>wrote:
> OS: HP/UX 11.11 IDS: 9.4FC3 (we are in the process of upgrading,
> but new server/db not ready for production yet)
>
> We started seeing these errors in our log Friday when users complained
> of not being able to connect. We checked back over time and there have
> been a few here and there, but Friday we had 50+ and today we have
100+
> and counting and are users are having a difficult time
connecting/using
> the system.
>
> The majority are:
> listener-thread: err = -25573: oserr = 233: errstr = : Network driver
> cannot accept a connection on the port.
>
> but we also see a few:
> listener-thread: err = -25580: oserr = 232: errstr = : System error
> occurred in network function.
>
> Nothing has changed on the server recently and network folks are
telling
> us that nothing has changed with them.
> Any ideas on what/where to look for/check/change to stop these errors?
>
> TIA,
> Randy
>
>
>
>
************************************************************************
*******
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--000e0cd25deeba53080497511690
************************************************************************
*******
Forum Note: Use "Reply" to post a response in the discussion forum.
Hmm. Does not sound like there should be a problem. Nasty the way reality
differs from expectation sometimes. I can only suggest bouncing the
instance and see if that helps.
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art@iiug.org)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.
On Mon, Dec 13, 2010 at 4:24 PM, Kennedy, Randy
<RKennedy@scottsdaleaz.gov>wrote:
> We only have 1 port and the highest maximum connections count from the
> log is 268 earlier this year and has been consistently between 210-250
> for the past couple of years.
>
> Our onconfig is set as:
> NETTYPE soctcp,4,150,NET
> NETTYPE ipcshm,1,25,NET>
> Thanks,
> Randy
>
> -----Original Message-----
> From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
> Art Kagel
> Sent: Monday, December 13, 2010 2:07 PM
> To: ids@iiug.org
> Subject: Re: network errors [22209]
>
> On HP/UX errno 232 is "Connection reset by peer" and 233 id "No buffer
> space
> available". The latter, more common error, sounds like the network
> drivers
> are being saturated. How many active concurrent connections do you have
> at
> the times when this is happening? A single poll threadr can handle up to
>
> about 1000 connections, however, IBM recommends configuring so that a
> thread
> handles no more than about 350 connections. One solution may be to
> increase
> the number of poll threads, however, it if is new connections that are
> the
> problem, then you probably should configure one or two additional ports
> (each with its own DBSERVERALIAS) so that there will be multiple
> listener
> threads listening for new connections and set things up so that the
> users
> are balanced across the two or three ports. You will have to bounce the
> server to implement this solution.
>
> BTW, in IDS 11.70 you can increase the number of listener threads on a
> single port in the ONCONFIG file or dynamically making configuring and
> resolving problems like this easier. IDS 11.70 also has additional
> features
> to reduce the overhead of connecting to the server which reduced the
> load so
> these problems are less common.
>
> Art
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> IIUG Board of Directors (art@iiug.org)
> Blog: http://informix-myview.blogspot.com/
>
> Disclaimer: Please keep in mind that my own opinions are my own opinions
> and
> do not reflect on my employer, Advanced DataTools, the IIUG, nor any
> other
> organization with which I am associated either explicitly, implicitly,
> or by
> inference. Neither do those opinions reflect those of other individuals
> affiliated with any entity with which I am affiliated nor those of the
> entities themselves.
>
> On Mon, Dec 13, 2010 at 3:44 PM, Kennedy, Randy
> <RKennedy@scottsdaleaz.gov>wrote:
>
> > OS: HP/UX 11.11 IDS: 9.4FC3 (we are in the process of upgrading,
> > but new server/db not ready for production yet)
> >
> > We started seeing these errors in our log Friday when users complained
>
> > of not being able to connect. We checked back over time and there have
>
> > been a few here and there, but Friday we had 50+ and today we have
> 100+
> > and counting and are users are having a difficult time
> connecting/using
> > the system.
> >
> > The majority are:
> > listener-thread: err = -25573: oserr = 233: errstr = : Network driver
> > cannot accept a connection on the port.
> >
> > but we also see a few:
> > listener-thread: err = -25580: oserr = 232: errstr = : System error
> > occurred in network function.
> >
> > Nothing has changed on the server recently and network folks are
> telling
> > us that nothing has changed with them.
> > Any ideas on what/where to look for/check/change to stop these errors?
>
> >
> > TIA,
> > Randy
> >
> >
> >
> >
> ************************************************************************
> *******
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --000e0cd25deeba53080497511690
>
> ************************************************************************
> *******
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--001636c5b15beb21060497519c7d
Unfortunately, already did that and it didn't help. Errors started
occurring again about 6 minutes after DB back online, at which time
there only about 30 connections.
Thanks,
Randy
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
Art Kagel
Sent: Monday, December 13, 2010 2:45 PM
To: ids@iiug.org
Subject: Re: network errors [22211]
Hmm. Does not sound like there should be a problem. Nasty the way
reality
differs from expectation sometimes. I can only suggest bouncing the
instance and see if that helps.
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art@iiug.org)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other
organization with which I am associated either explicitly, implicitly,
or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.
On Mon, Dec 13, 2010 at 4:24 PM, Kennedy, Randy
<RKennedy@scottsdaleaz.gov>wrote:
> We only have 1 port and the highest maximum connections count from the
> log is 268 earlier this year and has been consistently between 210-250
> for the past couple of years.
>
> Our onconfig is set as:
> NETTYPE soctcp,4,150,NET
> NETTYPE ipcshm,1,25,NET>
> Thanks,
> Randy
>
> -----Original Message-----
> From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
> Art Kagel
> Sent: Monday, December 13, 2010 2:07 PM
> To: ids@iiug.org
> Subject: Re: network errors [22209]
>
> On HP/UX errno 232 is "Connection reset by peer" and 233 id "No buffer
> space
> available". The latter, more common error, sounds like the network
> drivers
> are being saturated. How many active concurrent connections do you
have
> at
> the times when this is happening? A single poll threadr can handle up
to
>
> about 1000 connections, however, IBM recommends configuring so that a
> thread
> handles no more than about 350 connections. One solution may be to
> increase
> the number of poll threads, however, it if is new connections that are
> the
> problem, then you probably should configure one or two additional
ports
> (each with its own DBSERVERALIAS) so that there will be multiple
> listener
> threads listening for new connections and set things up so that the
> users
> are balanced across the two or three ports. You will have to bounce
the
> server to implement this solution.
>
> BTW, in IDS 11.70 you can increase the number of listener threads on a
> single port in the ONCONFIG file or dynamically making configuring and
> resolving problems like this easier. IDS 11.70 also has additional
> features
> to reduce the overhead of connecting to the server which reduced the
> load so
> these problems are less common.
>
> Art
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> IIUG Board of Directors (art@iiug.org)
> Blog: http://informix-myview.blogspot.com/
>
> Disclaimer: Please keep in mind that my own opinions are my own
opinions
> and
> do not reflect on my employer, Advanced DataTools, the IIUG, nor any
> other
> organization with which I am associated either explicitly, implicitly,
> or by
> inference. Neither do those opinions reflect those of other
individuals
> affiliated with any entity with which I am affiliated nor those of the
> entities themselves.
>
> On Mon, Dec 13, 2010 at 3:44 PM, Kennedy, Randy
> <RKennedy@scottsdaleaz.gov>wrote:
>
> > OS: HP/UX 11.11 IDS: 9.4FC3 (we are in the process of upgrading,
> > but new server/db not ready for production yet)
> >
> > We started seeing these errors in our log Friday when users
complained
>
> > of not being able to connect. We checked back over time and there
have
>
> > been a few here and there, but Friday we had 50+ and today we have
> 100+
> > and counting and are users are having a difficult time
> connecting/using
> > the system.
> >
> > The majority are:
> > listener-thread: err = -25573: oserr = 233: errstr = : Network
driver
> > cannot accept a connection on the port.
> >
> > but we also see a few:
> > listener-thread: err = -25580: oserr = 232: errstr = : System error
> > occurred in network function.
> >
> > Nothing has changed on the server recently and network folks are
> telling
> > us that nothing has changed with them.
> > Any ideas on what/where to look for/check/change to stop these
errors?
>
> >
> > TIA,
> > Randy
> >
> >
> >
> >
>
************************************************************************
> *******
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --000e0cd25deeba53080497511690
>
>
************************************************************************
> *******
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
>
>
************************************************************************
*******
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--001636c5b15beb21060497519c7d
************************************************************************
*******
Forum Note: Use "Reply" to post a response in the discussion forum.
I don't suppose you have extended support for that 9.40 server? It may be
time to call IBM.
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art@iiug.org)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.
On Mon, Dec 13, 2010 at 5:46 PM, Kennedy, Randy
<RKennedy@scottsdaleaz.gov>wrote:
> Unfortunately, already did that and it didn't help. Errors started
> occurring again about 6 minutes after DB back online, at which time
> there only about 30 connections.
>
> Thanks,
> Randy
>
> -----Original Message-----
> From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
> Art Kagel
> Sent: Monday, December 13, 2010 2:45 PM
> To: ids@iiug.org
> Subject: Re: network errors [22211]
>
> Hmm. Does not sound like there should be a problem. Nasty the way
> reality
> differs from expectation sometimes. I can only suggest bouncing the
> instance and see if that helps.
>
> Art
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> IIUG Board of Directors (art@iiug.org)
> Blog: http://informix-myview.blogspot.com/
>
> Disclaimer: Please keep in mind that my own opinions are my own opinions
> and
> do not reflect on my employer, Advanced DataTools, the IIUG, nor any
> other
> organization with which I am associated either explicitly, implicitly,
> or by
> inference. Neither do those opinions reflect those of other individuals
> affiliated with any entity with which I am affiliated nor those of the
> entities themselves.
>
> On Mon, Dec 13, 2010 at 4:24 PM, Kennedy, Randy
> <RKennedy@scottsdaleaz.gov>wrote:
>
> > We only have 1 port and the highest maximum connections count from the
>
> > log is 268 earlier this year and has been consistently between 210-250
>
> > for the past couple of years.
> >
> > Our onconfig is set as:
> > NETTYPE soctcp,4,150,NET
> > NETTYPE ipcshm,1,25,NET> >
> > Thanks,
> > Randy
> >
> > -----Original Message-----
> > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
> > Art Kagel
> > Sent: Monday, December 13, 2010 2:07 PM
> > To: ids@iiug.org
> > Subject: Re: network errors [22209]
> >
> > On HP/UX errno 232 is "Connection reset by peer" and 233 id "No buffer
>
> > space
> > available". The latter, more common error, sounds like the network
> > drivers
> > are being saturated. How many active concurrent connections do you
> have
> > at
> > the times when this is happening? A single poll threadr can handle up
> to
> >
> > about 1000 connections, however, IBM recommends configuring so that a
> > thread
> > handles no more than about 350 connections. One solution may be to
> > increase
> > the number of poll threads, however, it if is new connections that are
>
> > the
> > problem, then you probably should configure one or two additional
> ports
> > (each with its own DBSERVERALIAS) so that there will be multiple
> > listener
> > threads listening for new connections and set things up so that the
> > users
> > are balanced across the two or three ports. You will have to bounce
> the
> > server to implement this solution.
> >
> > BTW, in IDS 11.70 you can increase the number of listener threads on a
>
> > single port in the ONCONFIG file or dynamically making configuring and
>
> > resolving problems like this easier. IDS 11.70 also has additional
> > features
> > to reduce the overhead of connecting to the server which reduced the
> > load so
> > these problems are less common.
> >
> > Art
> >
> > Art S. Kagel
> > Advanced DataTools (www.advancedatatools.com)
> > IIUG Board of Directors (art@iiug.org)
> > Blog: http://informix-myview.blogspot.com/
> >
> > Disclaimer: Please keep in mind that my own opinions are my own
> opinions
> > and
> > do not reflect on my employer, Advanced DataTools, the IIUG, nor any
> > other
> > organization with which I am associated either explicitly, implicitly,
>
> > or by
> > inference. Neither do those opinions reflect those of other
> individuals
> > affiliated with any entity with which I am affiliated nor those of the
>
> > entities themselves.
> >
> > On Mon, Dec 13, 2010 at 3:44 PM, Kennedy, Randy
> > <RKennedy@scottsdaleaz.gov>wrote:
> >
> > > OS: HP/UX 11.11 IDS: 9.4FC3 (we are in the process of upgrading,
> > > but new server/db not ready for production yet)
> > >
> > > We started seeing these errors in our log Friday when users
> complained
> >
> > > of not being able to connect. We checked back over time and there
> have
> >
> > > been a few here and there, but Friday we had 50+ and today we have
> > 100+
> > > and counting and are users are having a difficult time
> > connecting/using
> > > the system.
> > >
> > > The majority are:
> > > listener-thread: err = -25573: oserr = 233: errstr = : Network
> driver
> > > cannot accept a connection on the port.
> > >
> > > but we also see a few:
> > > listener-thread: err = -25580: oserr = 232: errstr = : System error
> > > occurred in network function.
> > >
> > > Nothing has changed on the server recently and network folks are
> > telling
> > > us that nothing has changed with them.
> > > Any ideas on what/where to look for/check/change to stop these
> errors?
> >
> > >
> > > TIA,
> > > Randy
> > >
> > >
> > >
> > >
> >
> ************************************************************************
>
> > *******
> > > Forum Note: Use "Reply" to post a response in the discussion forum.
> > >
> > >
> >
> > --000e0cd25deeba53080497511690
> >
> >
> ************************************************************************
>
> > *******
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
> >
> >
> ************************************************************************
> *******
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --001636c5b15beb21060497519c7d
>
> ************************************************************************
> *******
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--20cf30433f00813d8a049752c488
Nope, just regular support and they won't talk to me about 9.4 unless it
is in reference to our upgrade to a newer version:)
Thanks,
Randy
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
Art Kagel
Sent: Monday, December 13, 2010 4:08 PM
To: ids@iiug.org
Subject: Re: network errors [22217]
I don't suppose you have extended support for that 9.40 server? It may
be
time to call IBM.
Art
Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art@iiug.org)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other
organization with which I am associated either explicitly, implicitly,
or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.
On Mon, Dec 13, 2010 at 5:46 PM, Kennedy, Randy
<RKennedy@scottsdaleaz.gov>wrote:
> Unfortunately, already did that and it didn't help. Errors started
> occurring again about 6 minutes after DB back online, at which time
> there only about 30 connections.
>
> Thanks,
> Randy
>
> -----Original Message-----
> From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
> Art Kagel
> Sent: Monday, December 13, 2010 2:45 PM
> To: ids@iiug.org
> Subject: Re: network errors [22211]
>
> Hmm. Does not sound like there should be a problem. Nasty the way
> reality
> differs from expectation sometimes. I can only suggest bouncing the
> instance and see if that helps.
>
> Art
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> IIUG Board of Directors (art@iiug.org)
> Blog: http://informix-myview.blogspot.com/
>
> Disclaimer: Please keep in mind that my own opinions are my own
opinions
> and
> do not reflect on my employer, Advanced DataTools, the IIUG, nor any
> other
> organization with which I am associated either explicitly, implicitly,
> or by
> inference. Neither do those opinions reflect those of other
individuals
> affiliated with any entity with which I am affiliated nor those of the
> entities themselves.
>
> On Mon, Dec 13, 2010 at 4:24 PM, Kennedy, Randy
> <RKennedy@scottsdaleaz.gov>wrote:
>
> > We only have 1 port and the highest maximum connections count from
the
>
> > log is 268 earlier this year and has been consistently between
210-250
>
> > for the past couple of years.
> >
> > Our onconfig is set as:
> > NETTYPE soctcp,4,150,NET
> > NETTYPE ipcshm,1,25,NET> >
> > Thanks,
> > Randy
> >
> > -----Original Message-----
> > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf
Of
> > Art Kagel
> > Sent: Monday, December 13, 2010 2:07 PM
> > To: ids@iiug.org
> > Subject: Re: network errors [22209]
> >
> > On HP/UX errno 232 is "Connection reset by peer" and 233 id "No
buffer
>
> > space
> > available". The latter, more common error, sounds like the network
> > drivers
> > are being saturated. How many active concurrent connections do you
> have
> > at
> > the times when this is happening? A single poll threadr can handle
up
> to
> >
> > about 1000 connections, however, IBM recommends configuring so that
a
> > thread
> > handles no more than about 350 connections. One solution may be to
> > increase
> > the number of poll threads, however, it if is new connections that
are
>
> > the
> > problem, then you probably should configure one or two additional
> ports
> > (each with its own DBSERVERALIAS) so that there will be multiple
> > listener
> > threads listening for new connections and set things up so that the
> > users
> > are balanced across the two or three ports. You will have to bounce
> the
> > server to implement this solution.
> >
> > BTW, in IDS 11.70 you can increase the number of listener threads on
a
>
> > single port in the ONCONFIG file or dynamically making configuring
and
>
> > resolving problems like this easier. IDS 11.70 also has additional
> > features
> > to reduce the overhead of connecting to the server which reduced the
> > load so
> > these problems are less common.
> >
> > Art
> >
> > Art S. Kagel
> > Advanced DataTools (www.advancedatatools.com)
> > IIUG Board of Directors (art@iiug.org)
> > Blog: http://informix-myview.blogspot.com/
> >
> > Disclaimer: Please keep in mind that my own opinions are my own
> opinions
> > and
> > do not reflect on my employer, Advanced DataTools, the IIUG, nor any
> > other
> > organization with which I am associated either explicitly,
implicitly,
>
> > or by
> > inference. Neither do those opinions reflect those of other
> individuals
> > affiliated with any entity with which I am affiliated nor those of
the
>
> > entities themselves.
> >
> > On Mon, Dec 13, 2010 at 3:44 PM, Kennedy, Randy
> > <RKennedy@scottsdaleaz.gov>wrote:
> >
> > > OS: HP/UX 11.11 IDS: 9.4FC3 (we are in the process of upgrading,
> > > but new server/db not ready for production yet)
> > >
> > > We started seeing these errors in our log Friday when users
> complained
> >
> > > of not being able to connect. We checked back over time and there
> have
> >
> > > been a few here and there, but Friday we had 50+ and today we have
> > 100+
> > > and counting and are users are having a difficult time
> > connecting/using
> > > the system.
> > >
> > > The majority are:
> > > listener-thread: err = -25573: oserr = 233: errstr = : Network
> driver
> > > cannot accept a connection on the port.
> > >
> > > but we also see a few:
> > > listener-thread: err = -25580: oserr = 232: errstr = : System
error
> > > occurred in network function.
> > >
> > > Nothing has changed on the server recently and network folks are
> > telling
> > > us that nothing has changed with them.
> > > Any ideas on what/where to look for/check/change to stop these
> errors?
> >
> > >
> > > TIA,
> > > Randy
> > >
> > >
> > >
> > >
> >
>
************************************************************************
>
> > *******
> > > Forum Note: Use "Reply" to post a response in the discussion
forum.
> > >
> > >
> >
> > --000e0cd25deeba53080497511690
> >
> >
>
************************************************************************
>
> > *******
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
> >
> >
>
************************************************************************
> *******
> > Forum Note: Use "Reply" to post a response in the discussion forum.
> >
> >
>
> --001636c5b15beb21060497519c7d
>
>
********************************************************************
On Mon, Dec 13, 2010 at 11:13 PM, Kennedy, Randy <RKennedy@scottsdaleaz.gov>wrote: > Nope, just regular support and they won't talk to me about 9.4 unless it > is in reference to our upgrade to a newer version:) > > Thanks, > Randy > No... If you have S&S up to date, you have support for 9.4. It is limited, but it's probably enough for your situation. Please check: https://www-304.ibm.com/support/docview.wss?uid=swg21439094 and then call support. There are lots of reports on the web about error 232/233 on HP-UX. Many mention lack of memory, performance issues etc. You can try a search about it... But I would suggest requesting IBM's help. It may be something outside Informix, but IBM should be able to help you get on the right direction. Regards. -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --0015174c371c0ec6d80497533ec0
We're on the phone with them now to see what help they will provide. Thanks, Randy -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Fernando Nunes Sent: Monday, December 13, 2010 4:41 PM To: ids@iiug.org Subject: Re: network errors [22219] On Mon, Dec 13, 2010 at 11:13 PM, Kennedy, Randy <RKennedy@scottsdaleaz.gov>wrote: > Nope, just regular support and they won't talk to me about 9.4 unless it > is in reference to our upgrade to a newer version:) > > Thanks, > Randy > No... If you have S&S up to date, you have support for 9.4. It is limited, but it's probably enough for your situation. Please check: https://www-304.ibm.com/support/docview.wss?uid=swg21439094 and then call support. There are lots of reports on the web about error 232/233 on HP-UX. Many mention lack of memory, performance issues etc. You can try a search about it... But I would suggest requesting IBM's help. It may be something outside Informix, but IBM should be able to help you get on the right direction. Regards. -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --0015174c371c0ec6d80497533ec0 ************************************************************************ ******* Forum Note: Use "Reply" to post a response in the discussion forum.
Randy To you get a solution to the problem. I am curious to know what the answer is just in case we ever hit probelm on our HP system
Hi, Any news? On Mon, Dec 13, 2010 at 11:44 PM, Kennedy, Randy <RKennedy@scottsdaleaz.gov>wrote: > We're on the phone with them now to see what help they will provide. > > Thanks, > Randy > > -----Original Message----- > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of > Fernando Nunes > Sent: Monday, December 13, 2010 4:41 PM > To: ids@iiug.org > Subject: Re: network errors [22219] > > On Mon, Dec 13, 2010 at 11:13 PM, Kennedy, Randy > <RKennedy@scottsdaleaz.gov>wrote: > > > Nope, just regular support and they won't talk to me about 9.4 unless > it > > is in reference to our upgrade to a newer version:) > > > > Thanks, > > Randy > > > > No... If you have S&S up to date, you have support for 9.4. It is > limited, > but it's probably enough for your situation. Please check: > > https://www-304.ibm.com/support/docview.wss?uid=swg21439094 > > and then call support. > There are lots of reports on the web about error 232/233 on HP-UX. Many > mention lack of memory, performance issues etc. > You can try a search about it... But I would suggest requesting IBM's > help. > It may be something outside Informix, but IBM should be able to help you > get > on the right direction. > > Regards. > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently... > > --0015174c371c0ec6d80497533ec0 > > ************************************************************************ > ******* > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --0015174bef2409a764049765c51e
No, IBM states it an HP (server) issue, so we are again dealing with HP. From some other diagnosis, it appears that there may be a system out there flooding our boxes with requests to an un-listened to port and it tying up the request for our database connections. The person with the packet sniffer is out sick today so we can't utilize that tool and we don't have tcpdump on the server either so... Thanks, Randy -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Fernando Nunes Sent: Tuesday, December 14, 2010 2:48 PM To: ids@iiug.org Subject: Re: network errors [22230] Hi, Any news? On Mon, Dec 13, 2010 at 11:44 PM, Kennedy, Randy <RKennedy@scottsdaleaz.gov>wrote: > We're on the phone with them now to see what help they will provide. > > Thanks, > Randy > > -----Original Message----- > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of > Fernando Nunes > Sent: Monday, December 13, 2010 4:41 PM > To: ids@iiug.org > Subject: Re: network errors [22219] > > On Mon, Dec 13, 2010 at 11:13 PM, Kennedy, Randy > <RKennedy@scottsdaleaz.gov>wrote: > > > Nope, just regular support and they won't talk to me about 9.4 unless > it > > is in reference to our upgrade to a newer version:) > > > > Thanks, > > Randy > > > > No... If you have S&S up to date, you have support for 9.4. It is > limited, > but it's probably enough for your situation. Please check: > > https://www-304.ibm.com/support/docview.wss?uid=swg21439094 > > and then call support. > There are lots of reports on the web about error 232/233 on HP-UX. Many > mention lack of memory, performance issues etc. > You can try a search about it... But I would suggest requesting IBM's > help. > It may be something outside Informix, but IBM should be able to help you > get > on the right direction. > > Regards. > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently... > > --0015174c371c0ec6d80497533ec0 > > ************************************************************************ > ******* > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > ************************************************************************ ******* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --0015174bef2409a764049765c51e ************************************************************************ ******* Forum Note: Use "Reply" to post a response in the discussion forum.
Have you tried a netstat -a? tcpdump can be found on the HP website, apparently included in a bundle of open source software. Not sure if it's easy to install, and you probably need some customer login. Regards. On Tue, Dec 14, 2010 at 10:24 PM, Kennedy, Randy <RKennedy@scottsdaleaz.gov>wrote: > No, IBM states it an HP (server) issue, so we are again dealing with HP. > >From some other diagnosis, it appears that there may be a system out > there flooding our boxes with requests to an un-listened to port and it > tying up the request for our database connections. The person with the > packet sniffer is out sick today so we can't utilize that tool and we > don't have tcpdump on the server either so... > > Thanks, > Randy > > -----Original Message----- > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of > Fernando Nunes > Sent: Tuesday, December 14, 2010 2:48 PM > To: ids@iiug.org > Subject: Re: network errors [22230] > > Hi, > Any news? > > On Mon, Dec 13, 2010 at 11:44 PM, Kennedy, Randy > <RKennedy@scottsdaleaz.gov>wrote: > > > We're on the phone with them now to see what help they will provide. > > > > Thanks, > > Randy > > > > -----Original Message----- > > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of > > Fernando Nunes > > Sent: Monday, December 13, 2010 4:41 PM > > To: ids@iiug.org > > Subject: Re: network errors [22219] > > > > On Mon, Dec 13, 2010 at 11:13 PM, Kennedy, Randy > > <RKennedy@scottsdaleaz.gov>wrote: > > > > > Nope, just regular support and they won't talk to me about 9.4 > unless > > it > > > is in reference to our upgrade to a newer version:) > > > > > > Thanks, > > > Randy > > > > > > > No... If you have S&S up to date, you have support for 9.4. It is > > limited, > > but it's probably enough for your situation. Please check: > > > > https://www-304.ibm.com/support/docview.wss?uid=swg21439094 > > > > and then call support. > > There are lots of reports on the web about error 232/233 on HP-UX. > Many > > mention lack of memory, performance issues etc. > > You can try a search about it... But I would suggest requesting IBM's > > help. > > It may be something outside Informix, but IBM should be able to help > you > > get > > on the right direction. > > > > Regards. > > > > -- > > Fernando Nunes > > Portugal > > > > http://informix-technology.blogspot.com > > My email works... but I don't check it frequently... > > > > --0015174c371c0ec6d80497533ec0 > > > > > ************************************************************************ > > > ******* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > > > > ************************************************************************ > ******* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > Fernando Nunes > Portugal > > http://informix-technology.blogspot.com > My email works... but I don't check it frequently... > > --0015174bef2409a764049765c51e > > ************************************************************************ > ******* > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --000e0cd1d2d815a678049766a034
We don't have a definitive answer, but it is a reverse DNS lookup problem. The majority of Informix's reverse lookup requests were timing out with the DNS servers so it would wait and repeat and if still failed, put them back to the bottom of the queue to try again. We added our users to the hosts file and connectivity was immediately restored. We are working with the DNS server admins to determine if it is something wrong with the DNS servers or if the problem is our HP/UX and/or Informix communication/setup with DNS. Thanks, Randy -----Original Message----- From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of KARL OLIVER Sent: Tuesday, December 14, 2010 12:21 PM To: ids@iiug.org Subject: Re: RE: network errors [22227] Randy To you get a solution to the problem. I am curious to know what the answer is just in case we ever hit probelm on our HP system ************************************************************************ ******* Forum Note: Use "Reply" to post a response in the discussion forum.
Bare in mind that Informix just calls an operating system function (typically gethostbyaddr() ) Regards. On Fri, Dec 17, 2010 at 7:42 PM, Kennedy, Randy <RKennedy@scottsdaleaz.gov>wrote: > We don't have a definitive answer, but it is a reverse DNS lookup > problem. The majority of Informix's reverse lookup requests were timing > out with the DNS servers so it would wait and repeat and if still > failed, put them back to the bottom of the queue to try again. > > We added our users to the hosts file and connectivity was immediately > restored. We are working with the DNS server admins to determine if it > is something wrong with the DNS servers or if the problem is our HP/UX > and/or Informix communication/setup with DNS. > > Thanks, > Randy > > -----Original Message----- > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of > KARL OLIVER > Sent: Tuesday, December 14, 2010 12:21 PM > To: ids@iiug.org > Subject: Re: RE: network errors [22227] > > Randy > To you get a solution to the problem. I am curious to know what the > answer is > just in case we ever hit probelm on our HP system > > ************************************************************************ > ******* > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Fernando Nunes Portugal http://informix-technology.blogspot.com My email works... but I don't check it frequently... --000e0cd1d2d82201f90497a243c1