17. 01. 2022 Massimo Giaimo Blue Team, SEC4U

An Evaluation of Elastic EDR with APT Simulator

We decided to carry out an evaluation of Elastic’s EDR using the APT Simulator (https://github.com/NextronSystems/APTSimulator) tool.

This tool is widely used within the cyber security community and is highly reputed, as it was developed by Florian Roth, who also created the Sigma Rule project.

APT Simulator is a Windows Batch script that uses a set of tools and output files to make a system look as if it has been compromised. In contrast to other adversary simulation tools, APT Simulator is designed to make the application as simple as possible. You don’t need to run a web server, database or any agents on a set of virtual machines. Just download the prepared archive, then extract and run the Batch file it contains as Administrator.

The following table shows the various test cases carried out by the tool along with their category.

Test CaseCategory
Collect Local FilesCollection
C2 ConnectsCommand and Control
DNS Cache 1 (Cache Injection)Command and Control
Malicious User Agents (Malware, RATs)Command and Control
Ncat Back Connect (Drop & Exec)Command and Control
WMI Backdoor C2Command and Control
LSASS Dump (with Procdump)Credential Access
Mimikatz 1 (Drop & Exec)Credential Access
WCE 1 (Eventlog entries)Credential Access
Active Guest Account AdminDefense Evasion
Fake System File (Drop & Exec)Defense Evasion
Hosts File (AV/Win Update blocks)Defense Evasion
Obfuscated JS DropperDefense Evasion
Obfuscation (RAR with JPG ext)Defense Evasion
Nbtscan Discovery (Scan & Output)Discovery
Recon Activity (Typical Commands)Discovery
PsExec (Drop & Exec)Execution
Remote Execution Tool (Drop)Execution
At JobPersistence
RUN Key Entry CreationPersistence
Scheduled Task CreationPersistence
StickyKey BackdoorPersistence
UserInitMprLogonScript PersistencePersistence
Web ShellsPersistence
WMI BackdoorPersistence

The details of the various tests carried out by APT Simulator are shown below.

Collection

Collect Local Files

  • Drops pwdump output to the working dir
  • Drops directory listing to the working dir

Command and Control

C2 Connects

  • Uses Curl to access well-known C2 servers

DNS Cache 1

  • Looks up several well-known C2 addresses to initiate DNS requests and get the addresses into the local DNS cache

Malicious User Agents

  • Uses malicious user agents to access web sites

Ncat Back Connect

  • Drops a PowerShell Ncat alternative to the working directory and runs it to back connect to a well-known attacker domain

WMI Backdoor C2

  • Uses Matt Graeber’s WMIBackdoor to contact a C2 at certain intervals

Credential Access

LSASS DUMP

  • Dumps LSASS process memory to a suspicious folder

Mimikatz-1

  • Dumps mimikatz output to working directory (fallback if other executions fail)
  • Runs a special version of mimikatz and dumps the output to working directory
  • Run Invoke-Mimikatz in memory (github download, reflection)

WCE-1

  • Creates Windows Eventlog entries that look as if WCE has been executed

Defense Evasion

Active Guest Account Admin

  • Activates Guest user
  • Adds Guest user to the local administrators

Fake System File

  • Drops suspicious executable with system file name (svchost.exe) in %PUBLIC% folder
  • Runs that suspicious program in %PUBLIC% folder

Hosts

  • Adds entries to the local hosts file (update blocker, entries caused by malware)

JS Dropper

  • Runs obfuscated JavaScript code with wscript.exe and starts decoded bind shell on port 1234/tcp

Obfuscation

  • Drops a cloaked RAR file with a JPG extension

Discovery

Nbtscan Discovery

  • Scans 3 private IP address class-C subnets and dumps the output to the working directory

Recon

  • Executes commands used by attackers to get information about a target system

Execution

PsExec

  • Dumps a renamed version of PsExec to the working directory
  • Runs PsExec to start a command line in LOCAL_SYSTEM context

Remote Execution Tool

  • Drops a remote execution tool to the working directory

Lateral Movement

No test cases yet

Persistence

At Job

  • Creates an at job that runs mimikatz and dumps credentials to file

RUN Key

  • Create a suspicious new RUN key entry that dumps “net user” output to a file

Scheduled Task

  • Creates a scheduled task that runs mimikatz and dumps the output to a file

Scheduled Task XML

  • Creates a scheduled task via XML file using Invoke-SchtasksBackdoor.ps1

Sticky Key Backdoor

  • Tries to replace sethc.exe with cmd.exe (a backup file is created)
  • Tries to register cmd.exe as a debugger for sethc.exe

Web Shells

  • Creates a standard web root directory
  • Drops standard web shells into that directory
  • Drops a GIF obfuscated web shell into that directory

UserInitMprLogonScript Persistence

  • Uses the UserInitMprLogonScript key to get persistence

WMI Backdoor

  • Uses Matt Graeber’s WMIBackdoor to kill local procexp64.exe when it starts

For our tests we used a Windows 10 client, with the latest available updates applied. The installation of the EDR Endpoint Security was carried out through the Fleet Server present within our SOC and a default policy was applied, without any particular exceptions.

Below are the results of the various tests performed and the possible detection capability by Elastic’s EDR:

Test CaseDetection Elastic EDR
Collect Local Files
C2 ConnectsOK
DNS Cache 1 (Cache Injection)OK
Malicious User Agents (Malware, RATs)OK
Ncat Back Connect (Drop & Exec)OK
WMI Backdoor C2OK
LSASS Dump (with Procdump)OK
Mimikatz 1 (Drop & Exec)OK
WCE 1 (Eventlog entries)OK
Active Guest Account AdminOK
Fake System File (Drop & Exec)OK
Hosts File (AV/Win Update blocks)OK
Obfuscated JS DropperOK
Obfuscation (RAR with JPG ext)OK
Nbtscan Discovery (Scan & Output)OK
Recon Activity (Typical Commands)OK
PsExec (Drop & Exec)Failed
Remote Execution Tool (Drop)Failed
At JobOK
RUN Key Entry CreationOK
Scheduled Task CreationOK
StickyKey BackdoorOK
UserInitMprLogonScript PersistenceOK
Web ShellsOK
WMI BackdoorOK

Once the tool has been run from a Powershell shell with administrative privileges, you can choose which tests to run:

During our tests we decided to start each single test individually, so as to be able to verify the events transmitted to Elastic from time to time.

As the tool suggests, if you want to replicate these tests within your environment, we recommend that you take a snapshot of the virtual machine used.

We now present the details of the detection capability of the various test categories.

Collection

This test is out of scope, as it is undetectable by EDR technology.

Command and Control

Result: Detected.

Evidence:

16:07:56.617
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:798   type:       diagnostic
16:07:56.617
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:799   name:       Delayed Execution via Ping
16:07:56.617
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:800   trusted:    false
16:07:56.617
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:807   alert sent: true
16:07:56.617
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] ResponseActions.cpp:86 Successfully killed process pid=13184, name=powershell.exe, result=0

Credential Access

Result: Detected

Evidence:

16:09:12.437
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:798   type:       diagnostic
16:09:12.437
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:799   name:       Delayed Execution via Ping
16:09:12.437
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:800   trusted:    false
16:09:12.437
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:807   alert sent: true
16:09:12.480
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] BulkQueueConsumer.cpp:224 Sent 1 documents to Elasticsearch
16:09:12.521
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] ResponseActions.cpp:86 Successfully killed process pid=8812, name=powershell.exe, result=0

Defense Evasion

Result: Detected

Evidence:

16:10:37.946
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:798   type:       diagnostic
16:10:37.946
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:799   name:       Execution from Unusual Directory
16:10:37.946
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:800   trusted:    false
16:10:37.946
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:807   alert sent: true
16:10:37.959
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] ResponseActions.cpp:86 Successfully killed process pid=13704, name=wscript.exe, result=0

Discovery

Result: Detected

Evidence:

16:11:50.130
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:798   type:       diagnostic
16:11:50.130
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:799   name:       Delayed Execution via Ping
16:11:50.130
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:800   trusted:    false
16:11:50.130
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:807   alert sent: true
16:11:50.178
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] BulkQueueConsumer.cpp:224 Sent 1 documents to Elasticsearch
16:11:50.341
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] ResponseActions.cpp:86 Successfully killed process pid=9352, name=WMIC.exe, result=0

Execution

Result: Not Detected

Evidence:

Persistence

Result: Detected

Evidence:

16:14:15.612
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:798   type:       diagnostic
16:14:15.612
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:799   name:       Delayed Execution via Ping
16:14:15.612
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:800   trusted:    false
16:14:15.612
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] RulesEngine.cpp:807   alert sent: true
16:14:15.649
elastic_agent.endpoint_security
[elastic_agent.endpoint_security][info] ResponseActions.cpp:86 Successfully killed process pid=12888, name=powershell.exe, result=0

From the point of view of the end user, the detections are displayed through the integration of the EDR in the notification system of the operating system.

While from the point of view of the SOC cyber security analysts it’s possible to obtain evidence of the various attack actions by activating the detection rule available by default within Elastic.

You can also create a dashboard with the details of the situation of the various endpoints present within the infrastructure:

Massimo Giaimo

Massimo Giaimo

Team Leader Cyber Security at Würth Phoenix

Author

Massimo Giaimo

Team Leader Cyber Security at Würth Phoenix

Leave a Reply

Your email address will not be published. Required fields are marked *

Archive