26. 06. 2019 Angelo Rosace Development, NetEye

Expanding Elastic Stack’s Set of Features

Last month, NetEye’s Elastic Stack received a much-awaited upgrade.

The upgrade consisted of granting NetEye users the possibility of having access to the full set of features that the Elastic Stack provides upon setting up an additional NetEye SIEM subscription.

Originally, the stack implemented on NetEye packaged the standard set of well-known ELK features: Elasticsearch, Logstash and Kibana. But X-Pack took everything up a notch. While making slight changes to the UI (in fact, some entries in the main menu of Kibana were added), the set of features included in the updated Elasticstack/X-Pack brings a large number of interesting functionalities with it.

In order to give users the option to best enable these functionalities, the R&D team didn’t simply provide the RPMs for the various modules of the Elastic Stack as they were released by the Elastic team. We started by undertaking a comparison between those packages already installed on NetEye and the new X-Pack packages. This was done because X-Pack ships with the files corresponding to the OSS features as well as those related to the non-free features. But the OSS ones were already present on NetEye and most probably had different paths or permissions than the others not already there. This situation required the R&D team to first download the X-Pack versions of the RPMs, define which filesbelonged to the OSS package and which to the actual X-Pack, resolve any conflicts that may have been created, and then package the files separately into two distinct RPMs (with the X-Pack one requiring the OSS RPM to be installed).

This way the installation of X-Pack resembles more of an upgrade than an actual full installation, while making it easier to extend the functionalities of the Elastic Stack when a customer buys a subscription.

To be more specific, these functionalities are: Alerting, Monitoring, Reporting, Graph, Machine Learning, Elasticsearch SQL, and Canvas.

(The X-Pack Security module is disabled since NetEye relies instead on Search Guard Compliance Edition for security).

To get a better understanding of the features, I’ll give a brief description of each one below:

Alerting

The Alerting module gives the user the possibility to set an alert on anything that one could query in Elasticsearch (e.g., the same user logged in from different locations, the Elasticsearch indexing rate has plummeted, etc.) You can also be notified in different ways: through Slack, email or something else.

Monitoring

The Monitoring module lets users understand what is happening in their Elastic Stack. It provides a set of dashboards that provides the user with all the information needed to keep Elastic Stack optimized.

Reporting

The Reporting module lets you generate reports of any Kibana visualization or dashboard. Report generation is as easy as pushing a button, and the resulting reports are both customizable and PDF-formatted.

Graph

The Graph module offers an overview of the connections within your data that can be found among the documents in your Elastic Stack. It offers an intuitive UI to let the user visualize these relations.

Machine Learning

The Machine Learning module automatically models the behavior of Elasticsearch data to identify issues faster, discover root causes, and reduce false positives.

Elasticsearch SQL

The Elasticsearch SQL module brings together the speed, scale and flexibility of Elasticsearch along with the familiar syntax of SQL. It lets you make the most out of your data queries with a well-known and easy to understand syntax.