In a previous blog post, I described how Elastic Stack fits within the High-Available cluster architecture of NetEye 4 and, in particular, how the correct configuration of the Quorum is mandatory to prevent losing your data or even developing inconsistencies.
With the upgrade to NetEye 4.8, we updated Elastic Stack to the new major version 7. This new version introduces a significant change in the management of the Elastic Stack Quorum.
In versions of Elastic Stack prior to 7, forming an Elastic Stack cluster required some manual configuration not only to discover nodes (discovery.zen.ping.unicast.hosts), but also to avoid critical situations such as split-brain conditions or data inconsistencies (discovery.zen.minimum_master_nodes). These configurations were mandatory and had to be continuously updated to reflect the current status of the architecture (e.g., if you decide to add a new master node, you must adjust minimum_master_nodes to avoid split-brain situations).
Starting with Elastic Stack 7, this configuration process was greatly simplified, so that you could form a cluster out-of-the-box without any special configuration. However, the basic configuration is not recommended for production systems.
For this reason, since NetEye 4.8 we used new configuration APIs to properly deploy safe and reliable Elastic Stack clusters:
discovery.seed_hosts: a list of nodes which a new node needs to contact the very first time when forming or joining a cluster
initial_master_nodes: this is used to ensure that before forming a cluster, all nodes listed in this configuration must be connected
As soon as the cluster is formed, Elastic Stack will take care of determining the number of master nodes, which will then take care of maintaining and securing the cluster. Thus all those configurations are automatically maintained and must not be adapted even if you want to scale up your master nodes.
Please note that manual configuration is still necessary if you want to remove pre-existing master nodes, by informing your Elastic Stack cluster that a node is going to be removed.
Hi, my name is Michele Santuari and I am a Telecommunication engineer felt in love with OpenFlow, the first attempt of centralized network management, provisioning, and monitoring. I embraced the Software Defined Networking approach to discover a passion for programming languages. Now, I am into Agile methodologies and crazy development process management.
Author
Michele Santuari
Hi, my name is Michele Santuari and I am a Telecommunication engineer felt in love with OpenFlow, the first attempt of centralized network management, provisioning, and monitoring. I embraced the Software Defined Networking approach to discover a passion for programming languages. Now, I am into Agile methodologies and crazy development process management.
Recently (in September 2023) NetEye integrated version 8.8 of the Elastic Stack, which is just one of many Elastic updates brought into NetEye 4. Since this Elastic update there was a major upgrade (from version 7.17) coming with many breaking Read More
The Fleet Management feature was automatically enabled with NetEye release 4.30, and with the current 4.31 version all the Elastic Stack packages will be upgraded to major version 8. These two milestones will permit us to centrally manage log ingestion Read More
Anyone who has joined the beautiful world of logging has collided, sooner or later, with the collection via syslog protocol. More than 40 years have passed since syslog was invented, and in that time there have been several attempts by Read More
The R&D Team is currently working on the integration of the new Elastic Fleet management tool in NetEye 4. Once Elastic Fleet is fully integrated in NetEye 4, all of the Log Management features currently supported will also need to Read More
The main goal of a monitoring system like NetEye is to alert and notify you when something noteworthy happens in your environment. All the logs coming in to NetEye SIEM can be analyzed, and could raise one or more alerts Read More