Does anyone have experience with error message "AS
The poster saw many "ASF Echo-Thread Server: asfcode = -25587 ... Network receive failed" messages in the online log and asked how to stop them via onconfig/sqlhosts. Responders agreed it's almost always caused by non-SQLI clients touching the Informix port - port scanners, firewalls/DOS probes, or monitoring tools (e.g. a Nagios check_tcp plugin) that connect and immediately close without sending data; an onmonitor-exit defect was also mentioned but ruled out. Advice: find/stop the scanning process, block the ports externally, or change monitoring to a real dbaccess query against sysmaster. With oserr=0 the message is considered harmless; the poster passed it to his sysadmins, and no confirmed fix is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0: errstr = : Network receive failed. We have lots of these, finderr explains it but I don't have a clue. What can we do to eliminate this error? Where can we modify in the onconfig or sqlhost or etc. Thanks in advance. finderr 25587 -25587 Network receive failed. A system call has failed. Perform a DISCONNECT and then check the status of the server. See the system administrator for assistance.
These are usually the result of some software that is not using Informix's SQLI protocol attempting to open the ports that Informix is listening on. This can be either DOS attacks from outside your firewall or your company's own port scans looking for offending ports. Block the server's ports at your external firewall to stop the former, get your sysadmins and security people to stop scanning the range of ports that the server is using to stop the latter. Art Art S. Kagel Advanced DataTools (www.advancedatatools.com) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, Nov 16, 2012 at 10:21 AM, Kern Doe <kern_doe@yahoo.com> wrote: > 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0: > errstr > = : Network receive failed. > We have lots of these, finderr explains it but I don't have a clue. What > can > we do to eliminate this error? Where can we modify in the onconfig or > sqlhost > or etc. Thanks in advance. > > finderr 25587 > -25587 Network receive failed. > A system call has failed. Perform a DISCONNECT and then check the > status of the server. See the system administrator for assistance. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --e89a8f2345075ff87704ce9e7aee
Original Post: 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0: errstr = : Network receive failed. We have lots of these, finderr explains it but I don't have a clue. What can we do to eliminate this error? Where can we modify in the onconfig or sqlhost or etc. Thanks in advance. finderr 25587 -25587 Network receive failed. A system call has failed. Perform a DISCONNECT and then check the status of the server. See the system administrator for assistance. Response: Well 1st off, I think there was a recent defect where every time you exited onmonitor it generated that error in your MSGPATH file. Secondly, the error means that the server tried to do a read off a fd set up for a network connection and the read failed. Possibly because the other end of the fd had closed the endpoint already. Since the oserr is saying it's 0, it does seem likely that it's just a badly processed shutdown request and the server tried to do a read after the other side had left, which then just causes the error to be printed but the thread would then likely shutdown it's endpoint and exit. I would suspect it's not really anything to be alarmed by (If the oserr was something non-zero it might be an indication of some network problem depending on what value it actually was), and again, it could be the above mentioned defect if you use onmonitor frequently and you have a version with that defect not fixed in it. Jacques Renaut IBM Informix Advanced Support APD Team
Thank you Jacques, read my comment below ________________________________ From: JACQUES RENAUT <jrenaut@us.ibm.com> To: ids@iiug.org Sent: Friday, November 16, 2012 9:57 AM Subject: Re: Does anyone have experience with error mes.... [28849] Original Post: 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0: errstr = : Network receive failed. We have lots of these, finderr explains it but I don't have a clue. What can we do to eliminate this error? Where can we modify in the onconfig or sqlhost or etc. Thanks in advance. finderr 25587 -25587 Network receive failed. A system call has failed. Perform a DISCONNECT and then check the status of the server. See the system administrator for assistance. Response: Well 1st off, I think there was a recent defect where every time you exited onmonitor it generated that error in your MSGPATH file. ****************************************** *****This is NOT the case relating "onmonitor"***** ****************************************** Secondly, the error means that the server tried to do a read off a fd set up for a network connection and the read failed. Possibly because the other end of the fd had closed the endpoint already. Since the oserr is saying it's 0, it does seem likely that it's just a badly processed shutdown request and the server tried to do a read after the other side had left, which then just causes the error to be printed but the thread would then likely shutdown it's endpoint and exit. I would suspect it's not really anything to be alarmed by (If the oserr was something non-zero it might be an indication of some network problem depending on what value it actually was), and again, it could be the above mentioned defect if you use onmonitor frequently and you have a version with that defect not fixed in it. ********************************************************* ******If it's not something to be alarmed, then I'm not worried********* ********************************************************* Jacques Renaut IBM Informix Advanced Support APD Team ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.
Thank you Art, I don't know for sure about "DOS attacks", but I forwarded your feedback to the sysadmin/netwk guy to evaluate. ________________________________ From: Art Kagel <art.kagel@gmail.com> To: ids@iiug.org Sent: Friday, November 16, 2012 9:32 AM Subject: Re: Does anyone have experience with error mes.... [28848] These are usually the result of some software that is not using Informix's SQLI protocol attempting to open the ports that Informix is listening on. This can be either DOS attacks from outside your firewall or your company's own port scans looking for offending ports. Block the server's ports at your external firewall to stop the former, get your sysadmins and security people to stop scanning the range of ports that the server is using to stop the latter. Art Art S. Kagel Advanced DataTools (http://www.advancedatatools.com/) Blog: http://informix-myview.blogspot.com/ Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves. On Fri, Nov 16, 2012 at 10:21 AM, Kern Doe <kern_doe@yahoo.com> wrote: > 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0: > errstr > = : Network receive failed. > We have lots of these, finderr explains it but I don't have a clue. What > can > we do to eliminate this error? Where can we modify in the onconfig or > sqlhost > or etc. Thanks in advance. > > finderr 25587 > -25587 Network receive failed. > A system call has failed. Perform a DISCONNECT and then check the > status of the server. See the system administrator for assistance. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > --e89a8f2345075ff87704ce9e7aee ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.
Hi,
we had this once when our sys-admin tried to check the port via a nagios
check_tcp plugin.
This always created such entries in online.log.
We have modified that to instead query a table in sysmaster of each instance
using a dbaccess
based check. This does not lead to such an error and makes sure the database
responds correctly
(the port may be up, but database is not responsive).
Maybe you have a similar situation there.
Marcus
----- Ursprüngliche Mail -----
Von: "Kern Doe" <kern_doe@yahoo.com>
An: ids@iiug.org
Gesendet: Freitag, 16. November 2012 19:37:24
Betreff: Re: Does anyone have experience with error mes.... [28851]
Thank you Art, I don't know for sure about "DOS attacks", but I forwarded your
feedback to the sysadmin/netwk guy to evaluate.
________________________________
From: Art Kagel <art.kagel@gmail.com>
To: ids@iiug.org
Sent: Friday, November 16, 2012 9:32 AM
Subject: Re: Does anyone have experience with error mes.... [28848]
These are usually the result of some software that is not using Informix's
SQLI protocol attempting to open the ports that Informix is listening on.
This can be either DOS attacks from outside your firewall or your company's
own port scans looking for offending ports. Block the server's ports at
your external firewall to stop the former, get your sysadmins and security
people to stop scanning the range of ports that the server is using to stop
the latter.
Art
Art S. Kagel
Advanced DataTools (http://www.advancedatatools.com/)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other organization with which I am associated either explicitly,
implicitly, or by inference. Neither do those opinions reflect those of
other individuals affiliated with any entity with which I am affiliated nor
those of the entities themselves.
On Fri, Nov 16, 2012 at 10:21 AM, Kern Doe <kern_doe@yahoo.com> wrote:
> 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0:
> errstr
> = : Network receive failed.
> We have lots of these, finderr explains it but I don't have a clue. What
> can
> we do to eliminate this error? Where can we modify in the onconfig or
> sqlhost
> or etc. Thanks in advance.
>
> finderr 25587
> -25587 Network receive failed.
> A system call has failed. Perform a DISCONNECT and then check the
> status of the server. See the system administrator for assistance.
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--e89a8f2345075ff87704ce9e7aee
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
Thanks Marcus, do you think having more ports may help to eliminate this
error? For this particular instance, it looks like there is only one port for
it.
For example, in the onconfig: DBSERVERALIASES --> instance1_tcp,
then in the sqlhosts: instance1_tcp ontlitcp hostABC instance1tcp
then in the /etc/services: instance1tcp 1530/tcp
Thanks in advance.
________________________________
From: Marcus Haarmann <marcus.haarmann@midoco.de>
To: ids@iiug.org
Sent: Friday, November 16, 2012 1:22 PM
Subject: Re: Does anyone have experience with error mes.... [28852]
Hi,
we had this once when our sys-admin tried to check the port via a nagios
check_tcp plugin.
This always created such entries in online.log.
We have modified that to instead query a table in sysmaster of each instance
using a dbaccess
based check. This does not lead to such an error and makes sure the database
responds correctly
(the port may be up, but database is not responsive).
Maybe you have a similar situation there.
Marcus
----- Ursprüngliche Mail -----
Von: "Kern Doe" <kern_doe@yahoo.com>
An: ids@iiug.org
Gesendet: Freitag, 16. November 2012 19:37:24
Betreff: Re: Does anyone have experience with error mes.... [28851]
Thank you Art, I don't know for sure about "DOS attacks", but I forwarded your
feedback to the sysadmin/netwk guy to evaluate.
________________________________
From: Art Kagel <art.kagel@gmail.com>
To: ids@iiug.org
Sent: Friday, November 16, 2012 9:32 AM
Subject: Re: Does anyone have experience with error mes.... [28848]
These are usually the result of some software that is not using Informix's
SQLI protocol attempting to open the ports that Informix is listening on.
This can be either DOS attacks from outside your firewall or your company's
own port scans looking for offending ports. Block the server's ports at
your external firewall to stop the former, get your sysadmins and security
people to stop scanning the range of ports that the server is using to stop
the latter.
Art
Art S. Kagel
Advanced DataTools (http://www.advancedatatools.com/)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other organization with which I am associated either explicitly,
implicitly, or by inference. Neither do those opinions reflect those of
other individuals affiliated with any entity with which I am affiliated nor
those of the entities themselves.
On Fri, Nov 16, 2012 at 10:21 AM, Kern Doe <kern_doe@yahoo.com> wrote:
> 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0:
> errstr
> = : Network receive failed.
> We have lots of these, finderr explains it but I don't have a clue. What
> can
> we do to eliminate this error? Where can we modify in the onconfig or
> sqlhost
> or etc. Thanks in advance.
>
> finderr 25587
> -25587 Network receive failed.
> A system call has failed. Perform a DISCONNECT and then check the
> status of the server. See the system administrator for assistance.
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--e89a8f2345075ff87704ce9e7aee
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
Original post: Thank you Jacques, read my comment below ________________________________ From: JACQUES RENAUT <jrenaut@us.ibm.com> To: ids@iiug.org Sent: Friday, November 16, 2012 9:57 AM Subject: Re: Does anyone have experience with error mes.... [28849] Original Post: 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0: errstr = : Network receive failed. We have lots of these, finderr explains it but I don't have a clue. What can we do to eliminate this error? Where can we modify in the onconfig or sqlhost or etc. Thanks in advance. finderr 25587 -25587 Network receive failed. A system call has failed. Perform a DISCONNECT and then check the status of the server. See the system administrator for assistance. Response: Well 1st off, I think there was a recent defect where every time you exited onmonitor it generated that error in your MSGPATH file. ****************************************** *****This is NOT the case relating "onmonitor"***** ****************************************** Secondly, the error means that the server tried to do a read off a fd set up for a network connection and the read failed. Possibly because the other end of the fd had closed the endpoint already. Since the oserr is saying it's 0, it does seem likely that it's just a badly processed shutdown request and the server tried to do a read after the other side had left, which then just causes the error to be printed but the thread would then likely shutdown it's endpoint and exit. I would suspect it's not really anything to be alarmed by (If the oserr was something non-zero it might be an indication of some network problem depending on what value it actually was), and again, it could be the above mentioned defect if you use onmonitor frequently and you have a version with that defect not fixed in it. ********************************************************* ******If it's not something to be alarmed, then I'm not worried********* ********************************************************* Jacques Renaut IBM Informix Advanced Support APD Team Response: Well, now that Art mentioned it, I do recall seeing some platforms where if you have some non-Informix application client (program, whatever you want to call it) that attempts to connect to the port that the server is listening on, and that program only connects but doesn't actually send any data after doing the connect, you can see those same type of errors. On some platforms it shows up like what you are seeing and on others the MSGPATH error will change and it will specifically say something about listener thread error (which makes it more obvious that there are bad incoming connections coming to the engine). So it could be some sort of network layer port probe to see if there is anything listening on the port which then doesn't send any data but just closes the connection after attempting to make it. Which would be different then a DOS attack, but could be some indication of something non-Informix client related pinging the port that the server is listening on. Which causes the errors to get reported to the MSGPATH file. Jacques Renaut IBM Informix Advanced Support APD Team
Hi Doe,
the error is not in the engine, you have to search to a process which is
connecting to the IP port
and then immediately closing the connection (like when you telnet on the port
and the close the
connection, you will get that error).
That could be a system like our nagios, which tests on the machine if the port
is open.
It is common use to check a tcp service for presence like that, connecting to
the port
and immediately closing the connection. That is what happened at our site.
There is probably nothing wrong with the DB and the TCP settings.
Marcus
----- Ursprüngliche Mail -----
Von: "Kern Doe" <kern_doe@yahoo.com>
An: ids@iiug.org
Gesendet: Freitag, 16. November 2012 20:35:29
Betreff: Re: Does anyone have experience with error mes.... [28854]
Thanks Marcus, do you think having more ports may help to eliminate this
error? For this particular instance, it looks like there is only one port for
it.
For example, in the onconfig: DBSERVERALIASES --> instance1_tcp,
then in the sqlhosts: instance1_tcp ontlitcp hostABC instance1tcp
then in the /etc/services: instance1tcp 1530/tcp
Thanks in advance.
________________________________
From: Marcus Haarmann <marcus.haarmann@midoco.de>
To: ids@iiug.org
Sent: Friday, November 16, 2012 1:22 PM
Subject: Re: Does anyone have experience with error mes.... [28852]
Hi,
we had this once when our sys-admin tried to check the port via a nagios
check_tcp plugin.
This always created such entries in online.log.
We have modified that to instead query a table in sysmaster of each instance
using a dbaccess
based check. This does not lead to such an error and makes sure the database
responds correctly
(the port may be up, but database is not responsive).
Maybe you have a similar situation there.
Marcus
----- Ursprüngliche Mail -----
Von: "Kern Doe" <kern_doe@yahoo.com>
An: ids@iiug.org
Gesendet: Freitag, 16. November 2012 19:37:24
Betreff: Re: Does anyone have experience with error mes.... [28851]
Thank you Art, I don't know for sure about "DOS attacks", but I forwarded your
feedback to the sysadmin/netwk guy to evaluate.
________________________________
From: Art Kagel <art.kagel@gmail.com>
To: ids@iiug.org
Sent: Friday, November 16, 2012 9:32 AM
Subject: Re: Does anyone have experience with error mes.... [28848]
These are usually the result of some software that is not using Informix's
SQLI protocol attempting to open the ports that Informix is listening on.
This can be either DOS attacks from outside your firewall or your company's
own port scans looking for offending ports. Block the server's ports at
your external firewall to stop the former, get your sysadmins and security
people to stop scanning the range of ports that the server is using to stop
the latter.
Art
Art S. Kagel
Advanced DataTools (http://www.advancedatatools.com/)
Blog: http://informix-myview.blogspot.com/
Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other organization with which I am associated either explicitly,
implicitly, or by inference. Neither do those opinions reflect those of
other individuals affiliated with any entity with which I am affiliated nor
those of the entities themselves.
On Fri, Nov 16, 2012 at 10:21 AM, Kern Doe <kern_doe@yahoo.com> wrote:
> 11/01/12 09:03:07 ASF Echo-Thread Server: asfcode = -25587: oserr = 0:
> errstr
> = : Network receive failed.
> We have lots of these, finderr explains it but I don't have a clue. What
> can
> we do to eliminate this error? Where can we modify in the onconfig or
> sqlhost
> or etc. Thanks in advance.
>
> finderr 25587
> -25587 Network receive failed.
> A system call has failed. Perform a DISCONNECT and then check the
> status of the server. See the system administrator for assistance.
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--e89a8f2345075ff87704ce9e7aee
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.