24. 12. 2023
Development, DevOps, Log-SIEM, NetEye
NetEye 4 is composed of various modules, such as the NetEye 4 Log Manager that houses Elastic Stack with Search Guard. Our vision for the future of the NetEye 4 Log Manager is shown in the following diagram:
Here you can see the various modules and technologies. For instance, you can see that we have included Beats (I talked about Beats in another blog post where I described how to use Icingabeat). But given its importance, I think Beats and its role within NetEye needs to be elaborated further.
Beats are an ecosystem of multiple lightweight components that send information to our NetEye 4 Log Manager. There are officially supported beats, and then community beats like Icingabeat. Here I’d like to talk in more detail about one of these community beats called Winlogbeat. As with all beats, Winlogbeat has just one task: to ship Windows event logs to Elasticsearch or Logstash. You can install it as a Windows service.
Winlogbeat reads from one or more event logs using Windows APIs, filters the events based on user-configured criteria, and then sends the event data to whatever outputs you have configured (in our case, Elasticsearch or Logstash). Winlogbeat monitors the event logs so that it can send new event data in a timely manner. The read position within each event log is also persisted on disk to allow Winlogbeat to resume after restarts.
Winlogbeat can capture event data from any event logs present on your Windows system. For example, you can capture events such as:
- Application events
- Hardware events
- Security events
- System events
Winlogbeat is an Elastic Beat based on the open source libbeat framework that permits anyone in the community to create their own Beat. Let’s see how you can use it.
I first selected a Windows Server to install Winlogbeat on and downloaded a copy compatible with the version of Elastic Stack in NetEye 4 Log Manager. I then installed it on my server and set up Winlogbeat by configuring its YAML configuration file. There I configured output to Logstash instead of Elasticsearch in order to retain the possibility to enrich or transform the data in the future, and selected the security event to send to my NetEye 4 Log Manager.
In NetEye 4 Log Manager I set up the firewall to only receive events from my Windows Server and Logstash, including the input and output ports for Winlogbeat.
Finally, I started the service in Windows Server and verified the Winlogbeat log in order to check that Winlogbeat was in fact sending events. It turns out it’s very fast: in just a few seconds, Winlogbeat sent thousands of events. In Kibana I set up the index winlogbeat-*, loaded the sample Winlogbeat dashboard, and voilà!
Typically, Beats are deployed on the edge of the network where deployments can easily span into the thousands. At this scale, monitoring and management become two key pillars of operational excellence. For instance, in Elastic Stack 6.2 there are monitoring Beats that provide deep visibility into the health and status of an entire deployment.
In Elastic Stack 6.5, Beats were introduced into central management, providing a centralized Kibana UI for managing and updating Beats configurations. This feature aims to drastically reduce users’ reliance on external configuration management tools like Ansible, Puppet, or Chef, enabling Beats configuration changes to be managed directly within the Elastic Stack (namely Kibana and Elasticsearch) and rolled out automatically to entire fleets of Beats.
Beats monitoring and Beats central management are in the X-Pack module of Elastic Stack. In my next blog, I’ll explore other Beats and how to correlate them.
Franco Federico
Hi, I’m Franco and I was born in Monza. For 20 years I worked for IBM in various roles. I started as a customer service representative (help desk operator), then I was promoted to Windows expert. In 2004 I changed again and was promoted to consultant, business analyst, then Java developer, and finally technical support and system integrator for Enterprise Content Management (FileNet). Several years ago I became fascinated by the Open Source world, the GNU\Linux operating system, and security in general. So for 4 years during my free time I studied security systems and computer networks in order to extend my knowledge. I came across several open source technologies including the Elastic stack (formerly ELK), and started to explore them and other similar ones like Grafana, Greylog, Snort, Grok, etc. I like to script in Python, too. Then I started to work in Würth Phoenix like consultant. Two years ago I moved with my family in Berlin to work for a startup in fintech(Nuri), but the startup went bankrupt due to insolvency. No problem, Berlin offered many other opportunities and I started working for Helios IT Service as an infrastructure monitoring expert with Icinga and Elastic, but after another year I preferred to return to Italy for various reasons that we can go into in person 🙂 In my free time I continue to dedicate myself to my family(especially my daughter) and I like walking, reading, dancing and making pizza for friends and relatives.
Author
Latest posts by Franco Federico
25. 03. 2024
APM, NetEye, Visual Synthetic Monitoring
Migration from Alyvix Server to Alyvix Service
10. 12. 2021
APM, NetEye, Real User Experience, Visual Synthetic Monitoring
NEP Alyvix – Ready for Use