18. 10. 2021 Mirko Bez NetEye, Unified Monitoring

Avoid Tornado Rules Repetition with a Map Post-modifier

In this post I’ll describe a concrete use case of the Tornado Map Modifier that will enable us to cover in a single rule many cases in both a user-friendly and performance-friendly way. This feature of Tornado allows us to avoid a common anti-pattern: the repetition of rules with minimal differences. This anti-pattern creates a set of rules that are difficult to maintain, because if something changes, we would have to change all of those rules. Moreover, it obliges Tornado to do redundant work.

From what I’ve seen in migration projects from Eventhandler to Tornado, this anti-pattern was extensively used in Eventhandler, because it was almost forced by Eventhandler’s rule structure. In Tornado instead, we can avoid this behavior by using variables and post-modifiers. Let’s see how.

A Concrete Use Case

A system sends messages with different priorities, which should be treated with different plugin exit statuses and different (notification) templates to allow for different handling of each problem. For instance, a job of priority 1 requires a critical status, a ticket and an SMS to be sent, but a job with lower priority requires only a ticket and a warning exit status, and so on.

A simplified example

For the sake of simplicity, let’s consider just 3 priorities: high, medium and low. An extremely simplified input that captures the essence of the problem can look like this:

{
  "priority": "High",
  "job": "My Cool Job",
  "other_relevant_info": "Very Interesting Job"
}

In the following table, I summarize the steps to be taken according to the different cases:

PriorityNotificationExit Status
HighSMS, Ticket2 (Critical)
MediumTicket1 (Warning)
LowTicket1 (Warning)

This problem reduces to choosing the right service template when creating a service and to setting the right exit status. Let’s say the templates are called mytemplate-1 (for high priority), my-template-2 (for medium priority) and my-template-3 for low priority.

The Anti-Pattern: Rule Repetition

At this point one common anti-pattern would be to create three different rules for the three different priorities. We can show the rule for the priority Medium:

The rules for the other priorities are practically identical. The only differences are the condition (high instead of medium), the exit_status and the service import. There are at least two problems with this approach: maintainability and performance.

With this configuration we have a maintainability problem because a minimal modification to a rule (e.g., changing the plugin_output) has consequences across all the rule priorities. Moreover, supporting additional priorities requires us to create additional rules.

Reducing the number of rules also helps in improving Tornado performance, because we are reducing its workload. Indeed, with three separate rules, Tornado has to make a comparison for each rule.

The Map Approach

We can elegantly solve this problem with the use of the Map Post-modifier (https://github.com/WuerthPhoenix/tornado/tree/develop/engine/matcher#the-with-clause—post-modifiers ), which is currently not fully supported in the GUI view.

The rule for all three services can be implemented like this:

Note that we are extracting two variables from the priority, service_template and exit_status, and using them in the action. The plugin-output and the host remain the same.

We don’t see it in the GUI, but I also added a post-modifier to the variables that look like this:

  "service_template": {
    "from": "${event.payload.priority}",
    "regex": {
     ...
    },
    "modifiers_post": [
      {
        "type": "Lowercase"
      },
      {
        "type": "Map",
        "mapping": {
          "high": "my-template-1",
          "medium": "my-template-2",
          "low": "my-template-3"
        },
        "default_value": null
      }
    ]
  }

Now, according to the priority, the right template is selected. Similarly for the exit_status I did the same:

"exit_status": {
    "from": "${event.payload.priority}",
    "regex": {
      ...
    },
    "modifiers_post": [
      {
        "type": "Lowercase"
      },
      {
        "type": "Map",
        "mapping": {
          "high": "2",
        },
        "default_value": "1"
      }
    ]
  }

Note that for the exit_status, we exploited the fact that Tornado supports a default value.

With this approach, I only have a single rule and I can extend/modify the priorities by simply extending the list of possibilities. Note also that the Map corresponds to the table we showed at the top of this blog post. As a Tornado user, I don’t need to care about the rule structure, just that my mapping is updated with my new requirements.

Conclusion

In this post, I described a lesser-known feature of Tornado: the Tornado Map Post-modifiers. This feature allows us to reduce the complexity of the rules and create maintainable and performance-friendly rules. Moreover, handy options like “default_value” potentially reduce the list size and allow us to cover yet-unknown cases. Finally, maps can be seen as tables and as such can also help inexperienced Tornado users extend Tornado rules. I cannot wait to see it fully supported in the GUI. Stay tuned!

Mirko Bez

Mirko Bez

Consultant at Würth Phoenix

Author

Mirko Bez

Consultant at Würth Phoenix

Leave a Reply

Your email address will not be published.

Archive