Icinga Timeout Values for Service Checks

[Brian Meyer]

Dear List,

I run Icinga on a VPS monitoring about 40 public services at my site.
I've increased the timeout value for most of my individual service
checks using the appropriate command line argument (45-60 seconds).
Yesterday checks were taking longer to complete due to some network
latency issues between the VPS and my site, exceeding the 45-60 timeout
and sending out a critical alert.. Now to fix this issue and increase my
timeouts to say 90 seconds would it be best for me to edit each
individual service definition, edit the check command directly, or
change the service_check_timeout directive in the main config file? I
know by default most plugins timeout after 10 seconds and by default
icinga enforces its own timeout on all service checks that is set to 30
seconds. What exactly does this mean? That by default Icinga will wait
40 seconds before sending a critical alert? Not exactly sure what the
service_check_timeout directive in the main config file actually does...

[Brian Meyer]

Dear List,

I run Icinga on a VPS monitoring about 40 public services at my site.
I've increased the timeout value for most of my individual service
checks using the appropriate command line argument (45-60 seconds).
Yesterday checks were taking longer to complete due to some network
latency issues between the VPS and my site, exceeding the 45-60 timeout
and sending out a critical alert.. Now to fix this issue and increase my
timeouts to say 90 seconds would it be best for me to edit each
individual service definition, edit the check command directly, or
change the service_check_timeout directive in the main config file? I
know by default most plugins timeout after 10 seconds and by default
icinga enforces its own timeout on all service checks that is set to 30
seconds. What exactly does this mean? That by default Icinga will wait
40 seconds before sending a critical alert? Not exactly sure what the
service_check_timeout directive in the main config file actually does...

[Michael Friedrich]

Post by Brian Meyer
Dear List,
I run Icinga on a VPS monitoring about 40 public services at my site.
I've increased the timeout value for most of my individual service
checks using the appropriate command line argument (45-60 seconds).
Yesterday checks were taking longer to complete due to some network
latency issues between the VPS and my site, exceeding the 45-60 timeout
and sending out a critical alert.. Now to fix this issue and increase my
timeouts to say 90 seconds would it be best for me to edit each
individual service definition, edit the check command directly, or
change the service_check_timeout directive in the main config file? I
know by default most plugins timeout after 10 seconds and by default
icinga enforces its own timeout on all service checks that is set to 30
seconds. What exactly does this mean? That by default Icinga will wait
40 seconds before sending a critical alert? Not exactly sure what the
service_check_timeout directive in the main config file actually does...

it sets an alarm timeout when forking a child process in order to
execute the check. if that does not return within that defined amount of
time, the parent process will issues a kill command on the child,
breaking out of the check process. you'll normally recognize that with a
custom output message the default timeout state is unknown, but you may
adjust that to critical in the main configuration file, if you require that.

there isn't any additional command object attribute for check timeouts
in icinga 1.x but will be provided in the new icinga 2 configuration
format for Check/Notification/EventCommands.

http://docs.icinga.org/icinga2/snapshot/#toc96 (note: url may change
once icinga 2 is final)
--
DI (FH) Michael Friedrich

mail: michael.friedrich-***@public.gmane.org
twitter: https://twitter.com/dnsmichi
jabber: dnsmichi-***@public.gmane.org
irc: irc.freenode.net/icinga dnsmichi

icinga open source monitoring
position: lead core developer
url: https://www.icinga.org