30. 11. 2024
Business Service Monitoring, NetEye, Unified Monitoring
16. 01. 2024
Patrick Zambelli
NetEye, Unified Monitoring
Icinga 2 DSL for Defining the Monitoring Status of Objects with Director
Today I want to present an Icinga 2-based monitoring use case where concepts of the powerful Icinga 2 DSL functional language come into play. The use case is based on mapping the status of a Host/Service Object via passive check results only. For this kind of use case, any accidental active status check could potentially overwrite the current status of the object.
Such use cases typically appear when monitoring incoming SNMP Traps, or mapping the status results provided by a third party Icinga 2 system. In these situations it’s important to avoid an accidental status change by an operator while performing for instance an active check or when clicking on the “Check now” action.
An experienced IcingaWeb2 administrator might think this should be solved via icingaweb2 role settings. Unfortunately as discussed in the open GitHub issue, it’s simply not possible to deny the “actions” permission for just some subset of hosts (i.e., the passive hosts) while simultaneously granting the same permission for other “active” objects. Therefore it’s necessary to prevent the status change on the passive hosts in some other way.
Any approach that implements a Plugin Script to calculate the status for a given Object and then returns that same status would be rather inefficient. In comparison, the Icinga 2 runtime variables provide all the required information, whereas the Icinga2 DSL allows them to be perfectly integrated in the monitoring configuration.
What the Icinga DSL language is
Icinga DSL (Domain Specific Language) is a powerful and flexible language used to define configuration objects and rules in Icinga. It allows users to write custom scripts and expressions to define monitoring behavior, notifications, and other aspects of the Icinga system. With Icinga DSL, users can create complex monitoring rules and configurations that would be difficult or even impossible to achieve using the standard configuration methods. It’s used within Icinga Director, a configuration tool for Icinga, to define advanced monitoring scenarios.
Some of the key advantages for using Icinga DSL include:
- Flexibility: Icinga DSL lets you create custom scripts and expressions that you can use to define monitoring behavior, notifications, and other Icinga system aspects. This provides a high level of flexibility and allows users to create complex monitoring rules and configurations you otherwise couldn’t with the standard configuration methods like the Director GUI.
- Powerful features: Icinga DSL includes a range of powerful features and functions, including support for variables, conditional statements, loops, and more. This allows users to create sophisticated monitoring rules and configurations that can adapt to changing conditions in real time. See the language reference guide
The Approach via the Icinga DSL Command
We start by defining a Host Command that can reflect the current status of a monitoring object onto itself in the case of an active check execution. The first step requires defining a command and argument value as a DLS instruction. When generating the DSL instruction with Director web UI, the surrounding “{{ }}” for defining the function is generated.
The function first resolves the host name of the check execution from the runtime variable. This can be accomplished by calling the macro function. In the next step we retrieve the desired host object by calling the “get_host()” method, and finally extract the desired attribute values from the returned object. That integer or string value is returned to the command variable.
To extract the current status code value, we can proceed with the following DSL Expression:
var strhost = macro("$host.name$")
var objhost = get_host(strhost)
var int_exit_status = objhost.last_check_result.state
return int_exit_statusSince the results of a host command should define both:
- Status code
- Output description
we slightly change the previous code snippet to the following:
var strhost = macro("$host.name$")
var objhost = get_host(strhost)
var str_output = objhost.last_check_result.output
return str_outputGiven that, here’s how it looks in Director:
And here’s what the preview looks like:
If we apply this host command via a host template to the host object(s), we’ll see this result:
And if we perform the Check now action we see:
Inspecting Icinga 2 Object Runtime Values and Evaluating Config Expressions in the Icinga CLI Console
All this seems to be rather magical, but luckily there’s a nice approach to debugging Icinga 2 runtime values via the icinga2 console. Take a look at the documentation for more details about using the CLI console.
Now I’ll give you a short example of how to connect to the console in your NetEye environment and inspect our relevant runtime variable. Connect to the console and then run the following steps:
- Extract the API credentials from the configuration file. To avoid any restrictions caused by lack of permissions, we connect to the API using the root user with permissions = “*”
# cat /neteye/shared/icinga2/conf/icinga2/conf.d/api-users.conf
/**
* The ApiUser objects are used for authentication against the API.
*/
object ApiUser "root" {
password = "your-secret-key"
permissions = [ "*" ]
}- Export the username and password into environment variables
# export ICINGA2_API_USERNAME=root
# export ICINGA2_API_PASSWORD= your-secret-key- Start a console session and query for the objects as with the DSL example from above
# icinga2-master console --connect 'https://root@localhost:5665/'
Icinga 2 (version: r2.14.0-1)
<1> => var h = get_host("dummy-selfStatus")
<2> => h.last_check_result
{
active = false
check_source = "neteye4.mydomain"
command = null
execution_end = 1704906173.819249
execution_start = 1704906173.819249
exit_status = 0.000000
output = "Host UP"
performance_data = null
previous_hard_state = 2.000000
schedule_end = 1704906173.819249
schedule_start = 1704906173.819249
scheduling_source = " neteye4.mydomain "
state = 0.000000
ttl = 0.000000
type = "CheckResult"
vars_after = {
attempt = 1.000000
reachable = true
state = 0.000000
state_type = 1.000000
}
vars_before = {
attempt = 1.000000
reachable = true
state = 2.000000
state_type = 1.000000
}
}- Now inspect the desired runtime value and voilà, you’ll see the same value that’s shown in the icingaweb2 UI:
<3> => h.last_check_result.state
0.000000
<4> => h.last_check_result.output
OK: Host UPEnjoy interacting with you Icinga2 objects! 😉
These Solutions are Engineered by Humans
Did you find this article interesting? Does it match your skill set? Our customers often present us with problems that need customized solutions. In fact, we’re currently hiring for roles just like this and others here at Würth Phoenix.
Patrick Zambelli
Project Manager at Würth Phoenix
After my graduation in Applied Computer Science at the Free University of Bolzano I decided to start my professional career outside the province. With a bit of good timing and good luck I went into the booming IT-Dept. of Geox in the shoe district of Montebelluna, where I realized how a big IT infrastructure has to grow and adapt to quickly changing requirements. During this experience I had also the nice possibility to travel the world, while setting up the various production and retail areas of this company. Arrived at Würth Phoenix I started developing on our monitoring solution NetEye. Today, in my position as Consulting an Project Manager I am continuously heading to implement our solutions to meet the expectation of your enterprise customers.
Author
Latest posts by Patrick Zambelli
23. 12. 2020
NetEye
Tornado – Getting in Action with Sample Rules
05. 05. 2020
NetEye, Unified Monitoring
Import Data Correlation and Automation with Icinga2 Director
20. 12. 2019
Icinga News, NetEye
One Year of Community for NetEye 4 Projects