EDR: The Great Escape - RomHack Training Review

Table of Contents

Romhack Logo

This course aims to provide a comprehensive understanding of the architecture of modern EDRs and their underlying Antivirus (AV) systems. It delves deeply into the complexity of modern EDRs, their structure, including the components responsible for real-time monitoring, data collection, and threat analysis.

[..]

50% of the course will be dedicated to hands-on labs showing how to translate the theory principles into practice. Labs are designed to provide flexibility in terms of complexity and include bonus tracks to ensure that you always feel engaged and have something interesting to explore and learn. This course is valuable not only for red team operators but also for blue team professionals. Blue team members can gain insights into how their detection systems may be bypassed, helping them enhance their security measures and stay one step ahead of potential threats. This course equips security professionals with a deep understanding of modern EDRs and their AV systems, enabling them to better simulate advanced threat scenarios, improve their evasion detection skills, and contribute to the overall enhancement of security within enterprise networks.

Throughout my years as a security analyst and consultant, I have attended numerous training courses and earned several certifications along the way. These trainings included both defensive (blue team) and offensive (red team) perspectives. Some of these training were labeled “Advanced”, but last week, I attended EDR: The Great Escape training by the two Crazy Italiene Doctors, and it was the first time in a long period that I felt that the course really deserved the label “Advanced” (this is of course very subjective, and for other participants, it might have been less demanding as it was for me). Silvio and Antonio are the founders of RETooling. You can find their latest research here (An unexpected journey into Microsoft Defender’s signature World), as we spoke several times about this research during the training.

In this post, I’ll review the training, discuss labs we worked on, and highlight key learning moments. As Silvio, one of the two teachers said a lot during the labs: Come on, it’s just a few lines of code and some printf statements! :)

Overview of Module 1

Filesystem Minifilter Driver

On the first day, we discussed the specific components of newer EDR systems which we will cover over the next four days, to find gaps in the implementations, and how to attack them. Within the course, we utilized the open-source EDR openedr as a reference architecture.

The first major block of the course was dedicated to minifilter drivers, where we were shown how to hunt for whitelists for paths or filenames that we could use to hide our implant or how the altitude of the filter manager could play tricks on the returned results. For me, the biggest “aha” experience in this module was the example of altitude sickness, a term coined by James Forshaw of Project Zero and explained in Hunting for Bugs in Windows Mini-Filter Drivers.

This is a bug class that is caused by the ordering of filter operations based on the assigned altitudes of the driver. For example, if you look at the list of filters from the fltmc command shown earlier in this blog post you’ll notice that WdFilter which is the real-time scanner for Windows Defender is at a much higher altitude than LUAFV which is the UAC file virtualization driver.

Leading, as you can imagine, to some really impressive bugs and exploits. In the lab, we were tasked to find a way to write a file to a folder protected by openedr, where we were able to put our newly acquired skills to the test.

Hooking

Nomen est omen. Every EDR monitors the API calls executed by a process in one way or another in order to detect suspicious and malicious behavior. In order to better understand unhooking, we first looked at various types of hooking techniques and then learned about unhooking and bypass techniques. In this module, the most significant learning for me was that syscalls can also be explicitly triggered via the VectoredExceptionHandler in order to bypass the hooking of EDR systems through this mechanism.

In the lab, we were tasked to remove a hook installed by the EDR through the APC injection technique. An amazing learning experience for me because we used WinDGB to analyze the hooks, and equipped with this knowledge we gained from the analysis, we wrote some C code which will first check if a hook is present and, if present, remove it. A clever way to put what we have learned into practice.

Module 2

ELAM and PPLs

ELAM stands for Early Launch Anti Malware, and although I knew the term and the technology in general, we were shown how to analyze the WdBoot driver with the Windows Kernel Debugger and how ELAM checks the information in the certificates for integrity since the file system is not yet available at boot time.

I taught about PPL, the Protected Process Light, in my FIRSTCON training this year, but even with some background in this topic, the level of presented details of the internals of PPLs was amazing.

Event Tracing for Windows

Certainly, one of the topics I was most looking forward to, as ETW (Event Tracing for Windows) is becoming increasingly important. Quote from the course: It was mainly introduced for capacity and performance analysis (and internal use!), but in the last few years, it has been adopted as a powerful tracing source for a wide range of AV/EDR solutions (e.g., Symantec, McAfee, Sophos etc.) and tools (Sysmon, wtrace etc.)

ETW

Figure 1: EtwSiloState (Source: blog.trailofbits.com)

The picture above is from the Trail of Bits blog, ETW internals for security research and forensics. In the lab, we then to mute ETW via the EtwUnregister API. The content was also great in this module - several times, I looked at the slides and thought - Wait, is this really possible?

Return Oriented Programming

I didn’t expect to find ROP techniques in this course, and I wasn’t aware that these techniques could be used for EDR evasions. This part of the course was all the more exciting, as was the accompanying lab:

In this lab, you must use ONLY ROP to allocate and copy a payload to a specified process. And from the provided solution: Instead of using a standard, well-known pattern of APIs from the injector code, we are using ROP to force the execution of our instructions directly inside the target process, making the whole process stealthier and much harder to detect by EDR solutions.

Module 3

Callbacks and Notify Routines

The module is all about a deep analysis of notify routine callbacks and object callbacks, as implemented by tools such as EDRSandBlast: EDRSandBlast is a tool written in C that weaponizes a vulnerable signed driver to bypass EDR detections (Notify Routine callbacks, Object Callbacks , and ETW TI provider) and LSASS protections.

From the defender (EDR vendor, Microsoft, SOC analysts looking at EDR’s telemetry, …) point of view, multiple indicators can be used to detect or prevent this kind of techniques. Since every action performed by the tool in kernel-mode memory relies on a vulnerable driver to read/write arbitrary content, driver loading events should be heavily scrutinized by the EDR product (or SOC analysts), and raise an alert at any uncommon driver loading, or even block known vulnerable drivers.

This brings us to the next topic…

Vulnerable Drivers

GMER

Figure 2: gmer.net - GMER is an application that detects and removes rootkits

As I work as an Incident Response Consultant, the topic of vulnerable drivers was naturally at the top of my list of interests, as we encounter them regularly in our IR or SOC cases. And I was not disappointed. We talked about Windows mitigations for vulnerable drivers, learned more about Windows Kernel Drivers, and of course how to exploit these (vulnerable) drivers. We analyzed various vulnerable drivers - and the corresponding offensive tools that exploit them - and then exploited a vulnerable driver in the lab to steal tokens from processes. 👏👏

Stack Spoofing

In general, it (the stack spoofing technique) could be combined with all other evasion techniques that need a clean and legitimate stack to make detections more complex.

We looked at various techniques such as Ekko, gargoyle, SilentMoonwalk, and CallStackMasker. Definitely one of the most challenging modules for me. In the lab, we integrated DeathSleep in our C2, A PoC implementation for an evasion technique to terminate the current thread and restore it before resuming execution, while implementing page protection changes during no execution.. Head over to the GitHub repository of DeathSleep to read more about the technical details of this implementation.

Local Privilege Escalation

I would also recommend the Local Privilege Escalation module to any SOC Analyst / IR Consultant, as there are an incredible number of relevant topics covered, including:

  • Access Tokens
  • Privileges
  • Logon Sessions - what exactly happens in the Kernel?
  • Integrity Levels
  • User Access Control (UAC)

We then implemented all the techniques we learned in this module to raise our privileges on our lab machine.

Windows Filtering Platform

WFP

Figure 3: High-Level Overview of the Windows Filtering Platform

A short introduction to the Windows Filtering Platform. Pavel Yosifovich wrote an interesting introduction about this subject here.

Memory Scanners

I really liked the concept of the whole class, where we picked apart well-known offensive techniques and tools like DripLoader, as well as given enough time to discuss strategies to detect malicious activities in our network, like moneta or pe-sieve

Module 4

.NET Internals

Day 4, afternoon. My brain is melted by the sheer amount of content from this unique class, but we still have some ground to cover:

  • .NET Header
  • Reflection
  • .NET Packer features
  • Dynamic Instrumentation

I knew about Reflection, and I knew about .NET packers, but what I learned here again will take me weeks to fully proceed and understand. For example, R2R stomping, which was novel to me, .. blew my mind. Additionally, the JIT Hooking used by MassLogger was impressive (see here).

On top of that, I learned A LOT about the .NET internals, ImplMap table, anyone?

Last lab of the course! This lab aims to teach you how to play with .NET IL (Intermediate language) in order to increase your ability to make obfuscated code. Specifically, we will substitute a Target Method IL with some random data to obfuscate the code when we use a proxy method that will call Runner in case of ÌnvalidProgamExecution. Using reflection, Runner will instantiate a DynamicMethod, getting the original method information from the exception handler and the IL with the relative metadata tokens passed by the calling method. 

Nice! The challenge was inspired by the 2022 Flare-On Challenge 8: backdoor (write-up here).

Conclusion

Me after four days: 🫠

The takeaway is another quote from the class: Research the techniques, and DO IT YOURSELF! :) Research until you find a blind spot, an exploit, or an angle to evade the EDR. Or, as a blue teamer, use that knowledge to search blindspots in your defense. I had a blast with this course, taught by two incredibly knowledgeable teachers.