Monitoring server status
Posted in 2007
Topics: General Discussion
Hello, we have three database servers that are accessed by several applications servers. Between them, there is a load balancer (BigIP from F5 company) that dispatches the load between these three database servers. We want to set up some kind of monitoring of the database servers to be sure they are up and running. When a database server fails for some reason, we want to make the load balancer aware of this situation so that it removes this database server from his pool and stops sending it queries. Currently we monitor every 3 seconds the TCP port to make sure it's in a listen status. This results in errors number 25580 and 27001 in the online.log file. Randomly, queries that used to take between 2 and 3 seconds can take up to 50 seconds. When, we deactivate the load balancer monitoring everything comes normal back (no error messages,queries takes between 2 and 3 seconds) . My question is: Is there another way to monitor the status of the database servers and, if one database server fails for some reason, provide this information to the load balancer so that it stops sending requests to this particular database server? Thanks
Use the ALARMPROGRAM ONCONFIG parameter to register an executable or script to
trap critical events and run send a message to the load balancer if the engine
reports a critical event problem. The list and description of event ids is in
the Administrator's Guide and Administrator's Reference. See my package
utils3_ak in the IIUG Software Repository for example script and executable
ALARMPROGRAM event handlers (these launch ontape or onbar to archive logical
logs as they fill and the executable also sends email notifications of other
critical events).
Art S. Kagel
----- Original Message -----
From: Mouhamadou Dia <ids@iiug.org>
At: 8/01 9:47:18
Hello,
we have three database servers that are accessed by several applications
servers. Between them, there is a load balancer (BigIP from F5 company) that
dispatches the load between these three database servers. We want to set up
some kind of monitoring of the database servers to be sure they are up and
running. When a database server fails for some reason, we want to make the
load balancer aware of this situation so that it removes this database server
from his pool and stops sending it queries. Currently we monitor every 3
seconds the TCP port to make sure it's in a listen status. This results in
errors number 25580 and 27001 in the online.log file. Randomly, queries that
used to take between 2 and 3 seconds can take up to 50 seconds. When, we
deactivate the load balancer monitoring everything comes normal back (no error
messages,queries takes between 2 and 3 seconds) . My question is: Is there
another way to monitor the status of the database servers and, if one database
server fails for some reason, provide this information to the load balancer so
that it stops sending requests to this particular database server?
Thanks
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
Thanks Art for your answer.
I'll have a look at your utils3_ak package
------------------------------------
Accovia Inc.
Mouhamadou Dia
Database Administrator
mdia@accovia.com
75 rue Queen
Montréal QC H3C2N6
tel: (514) 931-4433 #3137
------------------------------------
-----Message d'origine-----
De : ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] De la part de ART
KAGEL, BLOOMBERG/ 731 LEXIN
Envoyé : 1 août 2007 10:16
À : ids@iiug.org
Objet : Re:Monitoring server status [9669]
Use the ALARMPROGRAM ONCONFIG parameter to register an executable or script to
trap critical events and run send a message to the load balancer if the engine
reports a critical event problem. The list and description of event ids is in
the Administrator's Guide and Administrator's Reference. See my package
utils3_ak in the IIUG Software Repository for example script and executable
ALARMPROGRAM event handlers (these launch ontape or onbar to archive logical
logs as they fill and the executable also sends email notifications of other
critical events).
Art S. Kagel
----- Original Message -----
From: Mouhamadou Dia <ids@iiug.org>
At: 8/01 9:47:18
Hello,
we have three database servers that are accessed by several applications
servers. Between them, there is a load balancer (BigIP from F5 company) that
dispatches the load between these three database servers. We want to set up
some kind of monitoring of the database servers to be sure they are up and
running. When a database server fails for some reason, we want to make the
load balancer aware of this situation so that it removes this database server
from his pool and stops sending it queries. Currently we monitor every 3
seconds the TCP port to make sure it's in a listen status. This results in
errors number 25580 and 27001 in the online.log file. Randomly, queries that
used to take between 2 and 3 seconds can take up to 50 seconds. When, we
deactivate the load balancer monitoring everything comes normal back (no error
messages,queries takes between 2 and 3 seconds) . My question is: Is there
another way to monitor the status of the database servers and, if one database
server fails for some reason, provide this information to the load balancer so
that it stops sending requests to this particular database server?
Thanks
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
> Hello, > we have three database servers that are accessed by several > applications > servers. Between them, there is a load balancer (BigIP from > F5 company) that > dispatches the load between these three database servers. We > want to set up > some kind of monitoring of the database servers to be sure > they are up and > running. When a database server fails for some reason, we > want to make the > load balancer aware of this situation so that it removes this > database server > from his pool and stops sending it queries. Currently we > monitor every 3 > seconds the TCP port to make sure it's in a listen status. > This results in > errors number 25580 and 27001 in the online.log file. > Randomly, queries that > used to take between 2 and 3 seconds can take up to 50 > seconds. When, we > deactivate the load balancer monitoring everything comes > normal back (no error > messages,queries takes between 2 and 3 seconds) . My question > is: Is there > another way to monitor the status of the database servers > and, if one database > server fails for some reason, provide this information to the > load balancer so > that it stops sending requests to this particular database server? > > Thanks I'd suggest getting a fourth small server that sits on the client "side" of the F5 box, and periodically (maybe once a minute) does a test connection and database update (say, update a timestamp on a dummy table) on all three database servers. If it cannot connect / update, then maybe it can re-try a few times. If it is still unsuccessful, then it can send via ssh some commands to the F5 box to bring down and disable the appropriate pool. It may also have to kill all of the other connections in that pool, so that they will re-connect and go to one of the remaining pools. HTH, Paul M.