Inside Elastic Security Detection Rules: Internal Structure & Upgrade Mechanics
A Rule Is More Than a Query
In modern detection engineering, a rule is often misunderstood as just a query that triggers alerts. In reality, within Elastic Security, a detection rule is a structured, versioned, and lifecycle-managed object that goes far beyond simple query logic.
Understanding this structure is essential for anyone operating in a Security Operations Center (SOC), especially when managing large-scale detection content and updates.
A detection rule is composed of multiple layers, each serving a specific purpose:
- Identity & lifecycle metadata
- Execution configuration
- Detection logic
- Investigation metadata
- Integration dependencies
- Upgrade state & diff metadata
Rule identification
Elastic separates:
- id → Saved object instance (changes per installation)
- rule_id → Global identifier (stable across environments)
This design allows Elastic to:
- Apply safe upgrades
- Track the same prebuilt rule across clusters
- Compare local customizations
Versioning: version vs revision
Rule evolution is governed by two independent mechanisms:
version
- Controlled by Elastic (prebuilt content updates)
- Increases when detection logic or metadata changes
- Used for upgrade comparison
revision
- Increments on local edits
- Tracks administrative changes
- Does not imply detection logic changes
This distinction is particularly important in large SOC environments, where maintaining control over updates while preserving local modifications is critical.
Rule Source & Customization State
Each rule carries important context about its origin and modification status:
- The rule originates from Elastic content (external)
- Whether it was modified locally
- Whether a base version exists for diff comparison
This information forms the backbone of safe and controlled content upgrades. It allows teams to confidently apply updates without losing custom work.
Upgrade Engine: The diff Object
Elastic Security includes a built-in three-way merge system for rule updates. This is implemented through the diff object, which tracks changes at the field level.
Each field includes:
- base_version → original Elastic version
- current_version → your installed version
- target_version → new Elastic release
- merged_version → final resolved value
- merge_outcome → which side wins (BASE / CURRENT / TARGET)
- conflict → NONE or conflict type
This is effectively a three-way merge engine built into Elastic Security.
It allows:
- Automatic upgrades when no local changes exist
- Preservation of local customizations
- Conflict detection at field level
Detection Logic Is Only One Component
While detection logic is central, it is only one part of the rule.
Different rule types introduce distinct internal structures. For example:
- EQL rules rely on event correlation logic
- New Terms rules focus on identifying previously unseen values
Each type brings its own configuration and behavior, reinforcing that rules are not uniform objects but type-driven systems.
EQL Rule
New Terms Rule
Operational Metadata
Rules also include execution-related parameters such as:
- interval
- from
- to
- timestamp_override
- max_signals
- enabled
These directly impact:
- Query window calculation
- Alert generation boundaries
- Ingestion timestamp usage
- Alert volume control
Using timestamp_override: event.ingested ensures that late-arriving data is evaluated correctly, improving detection accuracy in delayed ingestion scenarios.
Investigation Enrichment
Detection rules also embed metadata designed to support analysts during investigations. These elements drive:
- Alert flyout context
- Timeline pivot fields
- Analyst triage experience
This is not part of detection logic itself—it is an intentional design to enhance analyst workflows and reduce investigation time.
Integration Awareness
Rules can declare dependencies on integrations and required ECS fields. This enables:
- Content compatibility validation
- Package version awareness
- Safer content lifecycle management
And required ECS fields:
What This Architecture Enables
From the structure alone, we can infer:
- Rules are saved objects managed by Kibana
- Upgrades use a three-way merge system
- Field-level diffing prevents destructive overwrites
- Local customizations are preserved
- Rule behavior is type-driven
- Detection and investigation metadata are decoupled
Elastic Security is not just running queries — it is managing detection-as-code with version control semantics.
Final Insight
The internal structure of a detection rule reveals Elastic Security’s core design philosophy:
Detection rules are versioned, mergeable, dependency-aware, and analyst-enriched content artifacts—not just saved searches.
Understanding this model provides:
- Safer upgrade strategies
- Better customization practices
- Stronger detection engineering discipline
Most importantly, it gives you full control over your detection lifecycle.
Alessio Dallaporta
Author
