During the NetEye User Group in November 2019, I presented the future vision of a new, completely distributed architecture to allow more flexibility, performance and scalability. In particular, a more flexible and scalable architecture is helpful for large environments such as Managed Service Provider infrastructures or for monitoring solutions as a service (i.e., NetEye 4 SaaS).
In this blog post, I present our R&D team’s current ongoing research activities in evaluating the challenge to migrate our service-based, unified monitoring solution (NetEye 4) to a modern fully distributed and scalable solution.
As you may know, NetEye 4 provides high-availability and load distribution thanks to the Red Hat cluster technology which distributes each tier of NetEye 4 across multiple physical servers. We intend to embrace the latest architectural trends to re-design the NetEye architecture to leverage orchestration-based platforms for containerized applications. The de facto standard of microservice management is Kubernetes, which provides deployment orchestration, service scaling, and high availability for applications.
After setting up a 3-node Kubernetes cluster in an on-premise high-availability configuration, we started creating a single container application to support the basic functionalities of NetEye.
As shown in the figure above, we created 3 containers:
NetEye running icingaweb2 and grafana UI, along with the icinga2 monitoring backend (in a future iteration we must split each component into a dedicated container)
MariaDB as the MySQL database
InfluxDB as the time series database
… but without persistent storage, we cannot distribute the load between the cluster nodes because as soon as a container is reallocated, all the data will be lost. For these reasons, we configured Rook with Ceph to create dynamic volumes to store data which needs to be persistent in the cluster:
We still have a lot of work to do, but this should be very helpful not only for learning, but also for trying out new architectural patterns for the NetEye of the future. If you are interested in these on-going activities, join us at the SFSCon conference in Bolzano: Andrea Avancini and I will talk about this and much more!
Hi, my name is Michele Santuari and I am a Telecommunication engineer felt in love with OpenFlow, the first attempt of centralized network management, provisioning, and monitoring. I embraced the Software Defined Networking approach to discover a passion for programming languages. Now, I am into Agile methodologies and crazy development process management.
Author
Michele Santuari
Hi, my name is Michele Santuari and I am a Telecommunication engineer felt in love with OpenFlow, the first attempt of centralized network management, provisioning, and monitoring. I embraced the Software Defined Networking approach to discover a passion for programming languages. Now, I am into Agile methodologies and crazy development process management.
Within our Research and Development team we maintain a set of both physical and virtual machines for many purposes: internal repository mirrors, CI/CD, testing, internal docker registry, etc... Maintaining them is demanding, especially considering that our infrastructure has grown over Read More
A customer with a NetEye installation to which I had added the nTop module asked me if I could integrate nTop's Grafana Dashboards, so they could view them by consulting NetEye's ITOA Menu. The installation and configuration of nTop for Read More
When I'm running a service which processes a lot of data, sooner or later I start to wonder: what is the service doing? What data is it processing? This also applies to our event processor Tornado. For the Tornado Engine, Read More
Ever since NetEye 4.11, the NetEye upgrade procedure has been automated with the neteye upgrade command. This command performs several checks to ensure that: The currently installed version is eligible to obtain the upgradeThe system is healthyThe latest OS and Read More
A frequent request I get is how to view the availability of hosts and/or services within a Grafana dashboard. Here I demonstrate how to do this for host availability. Host availability in Grafana ordered by availability To implement this, we Read More