In the past I’ve written in this blog post about Elastic Stack and its features.
Here I’d like to show you more in depth the functionality of Graph
analytics.
The Graph analytics features enable you to discover how items
in an Elasticsearch index are related. It’s possible to explore the connections
between indexed terms and see which connections are the most meaningful. This
can be useful in a variety of applications, from fraud detection to
recommendation engines. For example, graph exploration could help you uncover
website vulnerabilities that hackers are targeting so that you can harden your
website. Or, it’s possible to provide graph-based personalized recommendations
to your e-commerce customers. You can find the documentation about this
functionality here.
Suppose I have some data from a firewall that I want to
explore. I load this data in custom index that I call CEF (I chose this name
because the information is in this format) and I normalize the date using the
Elastic Common Schema (ECS).
The Elastic Common Schema (ECS) is an open source
specification developed with support from the Elastic user community. ECS
defines a common set of fields to be used when storing event data in
Elasticsearch, such as logs and metrics.
ECS specifies field names and Elasticsearch datatypes for
each field, and provides descriptions and example usage. ECS also groups fields
into ECS levels, which are used to signal how much a field is expected to be
present. You can learn more about ECS levels in Guidelines and
Best Practices.
Finally, ECS also provides a set of naming guidelines for adding your own
custom fields.
So first I open NetEye:
And then I open Log Analytics and arrive at Elastic Stack:
From here I open the Graph module:
Next I select my index “CEF” and select the field whose connection I’d like to explore in order to see insight or analytics about the security. In my case I choose the source and destination country, the city name, and the product VPN. It’s very simple to use and select fields – I can even choose the icon and color to associate with the component.
At this point I have to select the term that I want to use to explore the connection. I have data from a firewall, and I’d like to explore the data and the connection in drop, so I use the terms drop and reject. I then obtain this Graph:
With the following menu:
You could use this to:
Display additional vertices that connect to your graph, by clicking on the expand icon.
Display additional connections between the displayed vertices, by clicking on the link icon.
Explore a particular area of the graph, select the vertices you are interested in, and then click expand or link.
Step back through your changes to the graph, by clicking undo and redo.
Next I add an additional connection, after which the system selects further possible connections, and I find this resulting Graph in which cities of USA are placed.
I continue to add new vertices and more connections, and in the end I find a beautiful complex Graph:
So then I come back to the first simple Graph:
In this Graph I find confirmation that the firewall has been blocking many packets from Russia, the Netherlands and the United States, and I can investigate which city has a significant number of drops and rejects.
Hi, I’m Franco and I was born in Monza. Over the last 20 years I worked for IBM in various roles. I started as a customer service representative (help desk operator), then I was promoted to Windows expert. In 2004 I changed again and was promoted to consultant, business analyst, then Java developer, and finally technical support and system integrator for Enterprise Content Management (FileNet). Several years ago I became fascinated by the Open Source world, the GNU\Linux operating system, and security in general. And so in the last 4 years during my free time I studied security systems and computer networks in order to extend my knowledge. I came across several open source technologies including the Elastic stack (formerly ELK), and started to explore them and other similar ones like Grafana, Greylog, Snort, Grok, etc. I like to script in Python, too. In addition to studying in my free time I dedicate myself to my family (especially my little daughter) and I like walking, reading, dancing and making pizza for friends and relatives.
Author
Franco Federico
Hi, I’m Franco and I was born in Monza. Over the last 20 years I worked for IBM in various roles. I started as a customer service representative (help desk operator), then I was promoted to Windows expert. In 2004 I changed again and was promoted to consultant, business analyst, then Java developer, and finally technical support and system integrator for Enterprise Content Management (FileNet). Several years ago I became fascinated by the Open Source world, the GNU\Linux operating system, and security in general. And so in the last 4 years during my free time I studied security systems and computer networks in order to extend my knowledge. I came across several open source technologies including the Elastic stack (formerly ELK), and started to explore them and other similar ones like Grafana, Greylog, Snort, Grok, etc. I like to script in Python, too. In addition to studying in my free time I dedicate myself to my family (especially my little daughter) and I like walking, reading, dancing and making pizza for friends and relatives.
The R&D Team is currently working on the integration of the new Elastic Fleet management tool in NetEye 4. Once Elastic Fleet is fully integrated in NetEye 4, all of the Log Management features currently supported will also need to Read More
The main goal of a monitoring system like NetEye is to alert and notify you when something noteworthy happens in your environment. All the logs coming in to NetEye SIEM can be analyzed, and could raise one or more alerts Read More
In this blog post I will describe a potential use of Tornado to monitor events in near real-time, while keeping historical information about the received events. Use Case Often as a user I want to collect data from different sources, Read More
A bug has been discovered on NetEye modules logmanagement and SIEM. If affected, rsyslog directories on system might be created with wrong permissions causing Logstash to be unable to load log lines of some hosts inside Elasticsearch. Users might also Read More
The use case of this blog is about monitoring COVID-19 in Italy. The data used is public, and the source is the Protezione Civile (Italian Civil Protection Office), which updates the data every day after 18:00 on GitHub at the Read More