In a previous blog post, I described how Elastic Stack fits within the High-Available cluster architecture of NetEye 4 and, in particular, how the correct configuration of the Quorum is mandatory to prevent losing your data or even developing inconsistencies.
With the upgrade to NetEye 4.8, we updated Elastic Stack to the new major version 7. This new version introduces a significant change in the management of the Elastic Stack Quorum.
In versions of Elastic Stack prior to 7, forming an Elastic Stack cluster required some manual configuration not only to discover nodes (discovery.zen.ping.unicast.hosts), but also to avoid critical situations such as split-brain conditions or data inconsistencies (discovery.zen.minimum_master_nodes). These configurations were mandatory and had to be continuously updated to reflect the current status of the architecture (e.g., if you decide to add a new master node, you must adjust minimum_master_nodes to avoid split-brain situations).
Starting with Elastic Stack 7, this configuration process was greatly simplified, so that you could form a cluster out-of-the-box without any special configuration. However, the basic configuration is not recommended for production systems.
For this reason, since NetEye 4.8 we used new configuration APIs to properly deploy safe and reliable Elastic Stack clusters:
discovery.seed_hosts: a list of nodes which a new node needs to contact the very first time when forming or joining a clusterinitial_master_nodes: this is used to ensure that before forming a cluster, all nodes listed in this configuration must be connected
As soon as the cluster is formed, Elastic Stack will take care of determining the number of master nodes, which will then take care of maintaining and securing the cluster. Thus all those configurations are automatically maintained and must not be adapted even if you want to scale up your master nodes.
Please note that manual configuration is still necessary if you want to remove pre-existing master nodes, by informing your Elastic Stack cluster that a node is going to be removed.
External resources:
- https://www.elastic.co/guide/en/elasticsearch/reference/7.x/modules-discovery.html
- https://www.elastic.co/blog/a-new-era-for-cluster-coordination-in-elasticsearch
- https://www.elastic.co/guide/en/elasticsearch/reference/7.x/modules-discovery-adding-removing-nodes.html#modules-discovery-removing-nodes
Michele Santuari
Software Architect at Wuerth Phoenix
Hi, my name is Michele Santuari and I am a Telecommunication engineer felt in love with OpenFlow, the first attempt of centralized network management, provisioning, and monitoring. I embraced the Software Defined Networking approach to discover a passion for programming languages. Now, I am into Agile methodologies and crazy development process management.
Author
Latest posts by Michele Santuari
28. 12. 2020
Development
Research & Development – Spike (Part 4)
23. 12. 2020
Development
RPM/ISO Repository: Disk Space Optimization
05. 10. 2020
Development, NetEye
Research Activities: A Fully Distributed NetEye