Hey everyone! We played around a bit last time with our radar data to build a model that we could train outside Elasticsearch, loading it through Eland and then applying it using an ingest pipeline. But since our data is in the form of vectors, could we actually exploit Elasticsearch vector database functionality and perform…
Read MoreHi all, it’s been a while. I’m deeply sorry not to have sent out some blog posts lately, so now I’ll try to get back your trust by providing some useful information. Not only that, I’ll even go out of my comfort zone: instead of NetEye Core and monitoring strategies, I’ll talk about NetEye SIEM…
Read MoreElasticsearch limits the number of open shards per node with the max_shards_per_node cluster setting, which defaults to 1000. The limit on the total number of shards is then calculated from this setting with this formula: total_max_number_of_shards = cluster.max_shards_per_node * number of non-frozen data nodes If the total number of shards is reached either by a…
Read MoreSay you want to monitor logs coming into your Elasticsearch instance, and have it send data to your Monitoring Dashboard. I’ll show you how to do this with a practical example, in particular for an event coming from the Active Directory where a user is locked out, and the associated Domain Controller sends the event…
Read MoreFirst of all, I’ll briefly explain what the “Tornado” in NetEye actually is. Tornado is a Complex Event Processor that receives reports of events from data sources such as monitoring, email, and SNMP Traps, matches them against rules you’ve configured, and executes the actions associated with those rules, which can include sending notifications, logging to…
Read MoreIndex Lifecycle Management (ILM) policies constitute a fundamental component in Elasticsearch index management. They enable users to define the life stages of an index, determining when and how specific actions, such as transitioning from a “hot” to a “cold” state or deleting obsolete indices, should occur. ILM policies empower users to ensure the optimal distribution…
Read More
In my previous blog post, we saw how it’s possible to index some documents that we created by crawling our NetEye User Guide, then applying the ELSER model in Elasticsearch to create a bag of words for searching that takes into account the context of the various documents. Moreover, we also performed a simple query…
Read More
Once upon a time (in fact it was just a month ago, but it sounds more dramatic this way) I had the opportunity to attend a webinar about Vector Search, Generative AI, and modern NLP by the Elastic Team. One of the topics that was touched on during the webinar was ELSER , Elastic’s new…
Read MoreWith the introduction of the Composable Index Templates in Elastic, we at NetEye had to redesign the way index settings and mappings are applied to the indices generated by El Proxy. In this post I’ll explain: The solution explained in the remainder of the post was designed for El Proxy, but it may apply also…
Read MoreIn a previous article, we used NetEye and Elasticsearch to train a machine learning model able to classify documents about some collected radar signals, separating them into two categories (good vs bad), starting from an existing dataset. Afterwards, we applied it to new incoming documents using an Ingest Pipeline and the Inference Processor. Taking as…
Read MoreIn a previous article, we explored the Machine Learning capabilities of Elasticsearch, which allowed us to apply anomaly detection techniques to our data, and helped us discover some really interesting facts as a result of our analysis. But can we take that idea even further? For instance, could we use data we’ve already collected to…
Read MoreWith NetEye 4.22 we released a feature awaited for years: the ability to reach the Elasticsearch API externally, thanks to our NGINX proxy under NetEye. This new feature brings with it a lot of use cases, but what was turning over and over in my head was the automatic process for verification of the blockchains…
Read MoreThe R&D Team is currently working on the integration of the new Elastic Fleet management tool in NetEye 4. Once Elastic Fleet is fully integrated in NetEye 4, all of the Log Management features currently supported will also need to work with the Elastic Fleet. In particular, the integration of Elastic Fleet with the Log…
Read MoreWe have several customers using IBM AS400 whom we’ve helped in the past collect logs of system administrators under NetEye 3. Now with NetEye 4 we’ve improved log collection, making it compliant with the ECS standard and configuring a special internal port (5514) for NetEye to process these logs and syslog logs in general. Let’s…
Read MoreSeveral customers have asked us how they can collect DNS logs. In our solution, we proposed a Packetbeat Agent that allows you to collect data and send them to our centralized NetEye SIEM directly, or via a NetEye satellite. The Domain Name System (DNS) provides a hierarchy of names for computers and services on the…
Read More