30. 09. 2019 Franco Federico Log-SIEM

Filebeat and Log UI in NetEye

In a previous blog I explored beats such as Icingabeat and presented an overview of the new features present in NetEye since version 4.6.

I’d like to explore the following use case: collect some logs from Elasticsearch, Logstash, the operating system that hosts NetEye, and MySQL using beats (Filebeat), all in order to show the potential of Log UI.

I open Log Analytics and voilà:

Then I click on Add Log Data:

I have the various modules present and for each module there is a simple guide that accompanies me during the installation of the various components, with a link describing how to configure the various logs.

I start with the Elasticsearch log by clicking on the link.

Then I follow the various steps shown in the image.

In the file elasticsearch.yml I set the various paths of the Elasticsearch log, which in my case are in the folder /neteye/local/elasticsearch/log/.

At the end, after starting the Filebeat process, I can now verify my configuration with Module status.

The button Check data lets me verify whether data is arriving to my system and, in my case everything is okay.

I’m using the same procedure to configure the other logs (logstash, system and mysql).  Then I can explore that dashboard that I have in my system.  So I click on Dashboard and select my logs.

Here I’m only showing 3 dashboards. The others I’ll invite you to explore on your own.

I’ll show you just:

1) SSH login attempts

2) MySQL overview

3) Logstash logs

And now you might ask me, “Where do I find the Elasticsearch log?  And how can I correlate this log?”

The response is to click on Logs.

Using Logs is similar to tailing a log file in a shell, but with all of your logs from all of your systems available in a single console. The logs stream in, and the bottom of the view is the most recent record, just like a tail -f. By default, the Logs shows you all of the records from all of the logs that meet the configuration criteria.

When I’m working on an issue and decide that I don’t want all logs, from all services, all together (streaming by faster than anyone could possibly read), then I just change the interaction by typing in the search bar at the top. For example, if I only want to see errors on Logstash and the log message on Elasticsearch in order to correlate, I just start typing in the search bar and let the autocomplete help me find the right logs.

Another beautiful feature for me is that there is the possibility to view all logs, or a single log live by clicking on the button Stream live. Here I show how to view my Elasticsearch log in live mode.

It’s also possible to customize this view by clicking on the Custom button.

Franco Federico

Franco Federico

Hi, I’m Franco and I was born in Monza. Over the last 20 years I worked for IBM in various roles. I started as a customer service representative (help desk operator), then I was promoted to Windows expert. In 2004 I changed again and was promoted to consultant, business analyst, then Java developer, and finally technical support and system integrator for Enterprise Content Management (FileNet). Several years ago I became fascinated by the Open Source world, the GNU\Linux operating system, and security in general. And so in the last 4 years during my free time I studied security systems and computer networks in order to extend my knowledge. I came across several open source technologies including the Elastic stack (formerly ELK), and started to explore them and other similar ones like Grafana, Greylog, Snort, Grok, etc. I like to script in Python, too. In addition to studying in my free time I dedicate myself to my family (especially my little daughter) and I like walking, reading, dancing and making pizza for friends and relatives.

Author

Franco Federico

Hi, I’m Franco and I was born in Monza. Over the last 20 years I worked for IBM in various roles. I started as a customer service representative (help desk operator), then I was promoted to Windows expert. In 2004 I changed again and was promoted to consultant, business analyst, then Java developer, and finally technical support and system integrator for Enterprise Content Management (FileNet). Several years ago I became fascinated by the Open Source world, the GNU\Linux operating system, and security in general. And so in the last 4 years during my free time I studied security systems and computer networks in order to extend my knowledge. I came across several open source technologies including the Elastic stack (formerly ELK), and started to explore them and other similar ones like Grafana, Greylog, Snort, Grok, etc. I like to script in Python, too. In addition to studying in my free time I dedicate myself to my family (especially my little daughter) and I like walking, reading, dancing and making pizza for friends and relatives.

Leave a Reply

Your email address will not be published.

Archive