There’s a particular kind of irony in watching your observability stack become the thing you most need to observe. That’s more or less our day job, and it’s why we tend to be unusually deliberate about tooling decisions: The small ones have a habit of compounding over time. So when it came to choosing a…
Read MoreRarely has a title been more fitting: Transform metrics into alerts. It’s not just a description of what the system does – it’s also the exact name of the Elastic tool that makes it possible. Transforms, in their technical meaning, are the component we use to do precisely this: take a continuous stream of raw…
Read MoreIntroduction Managing Elasticsearch effectively – especially as environments grow in size and complexity – can quickly become a challenging task. Performance tuning, identifying the true root cause behind slowdowns, and optimizing resource allocation often demand specialized expertise and a significant investment in time. In enterprise ecosystems, where observability underpins critical services and reliability expectations are…
Read MoreIntroduction For on-premise Elasticsearch installations, performing a rolling restart across a cluster can be a time-consuming task, especially when dealing with large clusters. Rolling restarts are typically required when changing node configurations or upgrading the cluster to a new version. Elastic provides an official procedure to ensure service continuity during this process. However, after analyzing…
Read MoreWhile working with Kibana, we recently encountered a puzzling situation: queries involving the field event.original returned unexpected results. Let’s break down what happened, why it occurs, and how to identify other fields with similar behavior. The observed “strange behavior“ In fact, everything seems normal here: Now let’s assume you just want to see the documents…
Read MoreAs traffic to applications deployed on OpenShift grows, it’s essential to gain visibility into the flow of data entering your cluster. Monitoring this incoming traffic helps administrators maintain optimal performance, reduce security risks, and quickly resolve any emerging issues. Enabling Logging All traffic directed to an OpenShift Route is routed through a designated set of…
Read MoreWhen designing an Elasticsearch architecture, choosing the right storage is crucial. While NFS might seem like a convenient and flexible option, it comes with several pitfalls when used for hosting live Elasticsearch data (hot, warm, cold, and frozen nodes). However, NFS proves to be an excellent choice for storing snapshots and searchable snapshots. Here’s why….
Read MoreIn a previous blog post by one of my colleagues, we shared how we developed a powerful semantic search engine for our NetEye User Guide. This solution uses Elasticsearch in combination with machine learning models like ELSER to index and query our documentation. While the proof of concept (POC) worked great, there was a challenge…
Read MoreHello everyone! As you may remember, a topic I like to discuss a lot on this blog is the Proof of Concept (POC) about how we could enhance search within our online NetEye User Guide. Well, we’re happy to share with you how we’ve further developed the POC, specifically the process outlined in the first…
Read MoreWe all know that NetEye Upgrades are boring activities. Upgrading is important and useful because it brings you bug fixes and new features, but nonetheless it’s extremely expensive in terms of time. The most boring, tiring and lengthy part is when you restart NetEye Services; if you have the SIEM Module installed, the time spent…
Read MoreHey everyone! We played around a bit last time with our radar data to build a model that we could train outside Elasticsearch, loading it through Eland and then applying it using an ingest pipeline. But since our data is in the form of vectors, could we actually exploit Elasticsearch vector database functionality and perform…
Read MoreHi all, it’s been a while. I’m deeply sorry not to have sent out some blog posts lately, so now I’ll try to get back your trust by providing some useful information. Not only that, I’ll even go out of my comfort zone: instead of NetEye Core and monitoring strategies, I’ll talk about NetEye SIEM…
Read MoreElasticsearch limits the number of open shards per node with the max_shards_per_node cluster setting, which defaults to 1000. The limit on the total number of shards is then calculated from this setting with this formula: total_max_number_of_shards = cluster.max_shards_per_node * number of non-frozen data nodes If the total number of shards is reached either by a…
Read MoreSay you want to monitor logs coming into your Elasticsearch instance, and have it send data to your Monitoring Dashboard. I’ll show you how to do this with a practical example, in particular for an event coming from the Active Directory where a user is locked out, and the associated Domain Controller sends the event…
Read MoreFirst of all, I’ll briefly explain what the “Tornado” in NetEye actually is. Tornado is a Complex Event Processor that receives reports of events from data sources such as monitoring, email, and SNMP Traps, matches them against rules you’ve configured, and executes the actions associated with those rules, which can include sending notifications, logging to…
Read More