Workshop: Introduction to Adversary Emulation with Prelude Operator

Alexander Manners (Principal Security Engineer · Prelude Research), Kristopher Willis (Principal Security Engineer · Prelude Research)

Adversary Village @ c0c0n 2021 · Day 1 · Adversary Village

Overview

This workshop, "Introduction to Adversary Emulation with Prelude Operator," delivered by Alexander Manners and Kristopher Willis from Prelude Research, provides a comprehensive walkthrough of using the Prelude Operator platform for continuous defensive evaluation. The core premise revolves around the idea of "training the way you want to fight"—advocating for small, iterative, and repeatable actions to test and improve an organization's security posture against real-world threats. The speakers emphasize the importance of conducting these tests in production environments, albeit with careful consideration for potential impact, to truly assess the efficacy of existing defenses.

Watch on YouTube

Visual summary for Workshop: Introduction to Adversary Emulation with Prelude Operator by Alexander Manners, Kristopher Willis
Visual summary for Workshop: Introduction to Adversary Emulation with Prelude Operator by Alexander Manners, Kristopher Willis

Key moments

  1. 0:00 Introduction to Adversary Emulation & Workshop Goals
  2. 4:00 Foundations: Adversaries, Threats, and Cyber Threat Intelligence
  3. 11:00 Introduction to Prelude Operator Platform & Features
  4. 28:00 Hands-on: Deploying an Agent & First TTP Execution
  5. 40:00 Automating Attacks: Running a Pre-built Adversary Chain
  6. 51:00 Building a Custom Adversary Profile from CISA Alerts
  7. 92:00 Establishing Continuous Defensive Evaluation with Lessons Learned
  8. 97:00 Advanced: Deploying Cloud Redirectors for External Operations

Workshop: Introduction to Adversary Emulation with Prelude Operator

Speakers: Alexander Manners (Principal Security Engineer, Prelude Research), Kristopher Willis (Principal Security Engineer, Prelude Research)

Conference: Adversary Village @ c0c0n

YouTube: https://www.youtube.com/watch?v=ltlVI6sLZxY

Overview

This workshop, "Introduction to Adversary Emulation with Prelude Operator," delivered by Alexander Manners and Kristopher Willis from Prelude Research, provides a comprehensive walkthrough of using the Prelude Operator platform for continuous defensive evaluation. The core premise revolves around the idea of "training the way you want to fight"—advocating for small, iterative, and repeatable actions to test and improve an organization's security posture against real-world threats. The speakers emphasize the importance of conducting these tests in production environments, albeit with careful consideration for potential impact, to truly assess the efficacy of existing defenses.

The talk dives into the fundamental concepts of adversary emulation, starting from understanding various threat actors (from script kiddies to advanced persistent threats or APTs) and their tactics, techniques, and procedures (TTPs). It then transitions into a practical, hands-on demonstration of Prelude Operator, guiding attendees through agent deployment, executing individual TTPs, and orchestrating complex adversary chains. The ultimate goal is to enable security professionals to operationalize cyber threat intelligence (CTI) by simulating known adversary behaviors, thereby identifying gaps in detection and response capabilities and fostering a cycle of continuous improvement.

Prelude Operator positions itself as a versatile platform, integrating with established tools like Atomic Red Team and offering capabilities for local agent deployment, remote command and control (C2) through cloud redirectors, and the creation of custom TTPs. The workshop highlights how this approach allows defenders to go beyond theoretical security assessments, providing concrete, actionable insights into their systems' resilience against sophisticated attacks. By making adversary emulation accessible and repeatable, Prelude Operator aims to empower organizations to proactively strengthen their defenses.

Background

▶ Watch: Introduction to Adversary Emulation & Workshop Goals (0:00)

The foundation of effective cybersecurity lies in understanding and mitigating relevant threats. The speakers define an adversary broadly, encompassing any malicious actor from an unskilled "script kiddie" to a highly sophisticated APT. The key is that adversary emulation campaigns can be scaled in complexity to match the specific threat profile an organization faces. This assessment begins by asking critical questions: Who would target us? Why? What are their capabilities and TTPs? How would they attack?

To answer these questions, organizations must identify their crown jewels and centers of gravity—the most critical assets and core business components. Threat actors are likely to target these, and understanding the "most likely course of action" versus the "most deadly course of action" helps prioritize defensive efforts. This entire process is underpinned by Cyber Threat Intelligence (CTI). The speakers categorize CTI into three levels:

  • Strategic CTI: High-level threat actor objectives and targets.
  • Operational CTI: Ongoing campaigns and operations, revealing broader tactics.
  • Tactical CTI: Hands-on-keyboard actions, specific tools, and discrete techniques.

For open-source intelligence gathering, the CISA Alerts page (from the Cybersecurity and Infrastructure Security Agency) is recommended as a valuable resource for understanding current threats across various industries. Once relevant threat intelligence is gathered, the next step is to select an attack platform. The C2 Matrix lists over 90 command and control (C2) frameworks, ranging from open-source options like Sliver and Caldera to commercial platforms. For this workshop, the focus is exclusively on Prelude Operator.

The overarching philosophy presented is the concept of a continuous defensive evaluation process. This involves performing small, iterative actions over time to incrementally improve system defenses. The speakers emphasize "training the way you want to fight," which implies testing in production environments where the actual defenses are deployed. The most crucial aspect is to "keep it simple," taking small, measurable steps and consistently assessing their efficacy to drive continuous improvement.

Key Findings

▶ Watch: Introduction to Prelude Operator Platform & Features (11:00)

The central "finding" or contribution of this talk and the Prelude Operator platform is the operationalization of adversary emulation as a practical, repeatable, and continuous process for defensive evaluation. The speakers demonstrate that by leveraging readily available cyber threat intelligence (CTI), organizations can systematically emulate components of APT kill chains to proactively identify and remediate defensive gaps.

Key findings highlighted by the workshop include:

  1. Accessibility of Adversary Emulation: Prelude Operator simplifies the complex task of adversary emulation, making it accessible to a broader range of security professionals. Its intuitive interface and pre-built TTPs (Tactics, Techniques, and Procedures) allow users to quickly deploy agents and execute attack simulations without extensive red teaming expertise.
  2. Operationalizing CTI: The platform provides a direct mechanism to translate CTI into actionable defensive tests. By mapping adversary behaviors (e.g., from CISA alerts) to specific MITRE ATT&CK techniques and then to executable TTPs within Operator, security teams can validate their defenses against known threats.
  3. Continuous Defensive Evaluation: The emphasis on "small, iterative actions" and "closing the loop" (lessons learned) establishes a framework for ongoing security improvement. This shifts security from a reactive to a proactive stance, allowing organizations to continually tune their detections and response capabilities.
  4. Flexibility and Customization: Prelude Operator supports a diverse range of agents (Go, Python, Nim, JavaScript) across multiple operating systems (Windows, macOS, Linux). It also empowers users to create and modify custom TTPs, integrate with external tools (like Atomic Red Team), and leverage cloud infrastructure for realistic, internet-facing campaigns.
  5. Measurable Outcomes: The platform maintains a detailed log of all executed actions and their results, including automatically extracted variables. This comprehensive logging facilitates post-execution analysis, enabling security teams to objectively assess their detection rates and refine their security controls based on concrete evidence.

In essence, the talk finds that effective security improvement stems from consistent, data-driven testing against realistic adversary behaviors, a process that Prelude Operator is designed to facilitate.

Technical Deep Dive

▶ Watch: Automating Attacks: Running a Pre-built Adversary Chain (40:00)

Prelude Operator is presented as a comprehensive platform for managing and executing adversary emulation campaigns. The user interface is organized into several key tabs: Operate, Train, Editor, Reports, Plugins, and Docs, alongside an account management section.

The Operate tab serves as the central hub for managing agents and ranges. A range acts as a campaign container, grouping multiple agents for coordinated operations. Users can create chains, which are sequences of procedures (TTPs) designed to accomplish specific objectives, ultimately forming an adversary profile.

Operator supports various agent types to ensure broad compatibility:

  • Numa: A Go-based agent, considered the "bread and butter," supporting Windows, macOS, and Linux.
  • Schism: A Python-based agent.
  • Nicodemus: A community-driven Nim implant.
  • Hush: A JavaScript JXA agent, primarily for macOS with future cross-platform plans.
  • Numa Extended: An upgraded, framework-style agent offering advanced configuration options for professional licenses, including encryption and network settings.

Agents can operate in two modes: interactive for real-time command execution or beaconing, where agents periodically check in with the C2 server for new tasks. The platform provides detailed logging of all agent actions, categorizing them as green (successful/running) or red (failed/disconnected).

The Editor tab is crucial for managing TTPs, the discrete actions an adversary might perform. TTPs are tagged (e.g., "Conti" for ransomware, "Docker" for container-related techniques) and mapped to the MITRE ATT&CK framework (tactics and techniques). Each TTP specifies its executor (e.g., sh, python, bash, zsh for Linux) and the command to be executed. The editor allows users to view, edit, deploy, and create new TTPs, supporting template-like variables (e.g., #{variable_name}) that can be dynamically rendered during execution. A critical feature is the destructive tag, which prevents TTPs that could cause irreversible system changes (e.g., deleting a home directory) from being executed without explicit user confirmation.

Plugins extend Operator's functionality:

  • Atomic Red Team: Imports all Atomic Red Team TTPs.
  • Navigator: Displays MITRE ATT&CK Navigator layers.
  • Toolbox: Clones and tracks GitHub repositories, keeping red team tools up-to-date.
  • Cloud/Connect: A significant feature enabling the deployment of redirectors (e.g., HAProxy) and test ranges in cloud environments like AWS and Google Cloud Platform (GCP). This allows agents to communicate over the internet, facilitating more realistic external attack simulations.
  • Planner: An engine for ordering TTP execution within an adversary.
  • Publisher: Sends agent results to external databases, SIEMs, or chat rooms.

Network settings can be configured to expose specific ports (TCP, UDP, HTTP, reverse shell) and interfaces (localhost or 0.0.0.0 for remote access). The platform also supports an offline mode for air-gapped networks.

A core technical concept is the use of facts or variables. TTPs can be templated with placeholders like #{ipt1016} or #{execution.remote.ip}. Operator's automation engine can automatically extract information (e.g., IP addresses from ifconfig output) and store them as facts, which are then used to populate subsequent TTP commands. Users can also manually inject custom facts (key-value pairs) into agents to provide necessary parameters for TTP execution, such as a remote IP address and port for a reverse shell.

Payload management allows users to view the contents of scripts like powercat.ps1. While built-in payloads are read-only to maintain version integrity, users can incorporate custom payloads by modifying TTPs or importing external repositories, ensuring that all tools are tracked and potentially updated. The ability to import TTPs from local directories or remote GitHub URLs facilitates team collaboration and version control for custom adversary profiles.

Demo / Proof of Concept

▶ Watch: Building a Custom Adversary Profile from CISA Alerts (51:00)

The workshop provided a comprehensive, hands-on demonstration of Prelude Operator's capabilities, guiding participants through various stages of adversary emulation.

  1. Agent Installation and Connection:
  • Attendees were instructed to download the Numa agent (version 1.2) for their respective operating systems (Windows, macOS, Linux) from prelude.org.
  • For Windows users, a workaround for Windows Defender flagging Numa was provided: compiling the agent from source using garble build from the Numa GitHub repository.
  • The agent was connected via the command line using the --debug flag, defaulting to localhost for initial setup. Remote connections were demonstrated later using the --address flag.
  • Once connected, the agent appeared in the "red range" (the default range) within the Operator UI.
  1. Running a Single TTP:
  • From the Editor tab, Alex demonstrated deploying a single discovery TTP: "snag broadcast ip" (mapped to MITRE ATT&CK T1016).
  • The TTP executed ifconfig | grep broadcast on a macOS system.
  • The results were immediately visible in the UI, showcasing the command output and, crucially, the automatic extraction of variables like ipt1016 (IP addresses) from the output. This highlighted Operator's ability to gather and leverage intelligence from executed TTPs.
  • The repeatability of this process was emphasized, allowing defenders to run the same test multiple times to tune detections.
  1. Deploying an Adversary Chain (File Hunter):
  • The concept of chains (adversary profiles) was introduced, which link multiple TTPs into an automated sequence.
  • The "File Hunter" chain was demonstrated, designed to perform local discovery, create a staging directory (mkdir -p /tmp/stage), copy files into it, and then compress the directory for exfiltration.
  • This demo illustrated how the variables extracted from earlier TTPs (e.g., directory t1074 for the staging path) are seamlessly passed to subsequent TTPs in the chain (e.g., cp commands).
  • The output showed the creation of the directory, finding recent files (e.g., the numa darwin agent itself), copying them, and finally compressing them into a .tar.gz archive.
  1. Building a Custom APT40 Adversary Profile:
  • Using a CISA alert on APT40 as a source of CTI, the speakers identified specific techniques used by the threat actor, such as archiving, encrypting, staging data, exfiltration over C2, use of PowerShell, and credential dumping (Mimikatz).
  • A new adversary profile named "basic apt40" was created.
  • TTPs were added to this profile from the editor:
  • file.ioxfill (for exfiltration to file.io).
  • create remote powershell with powercat.
  • install pstools.
  • invoke mimikatz dump creds.
  • get keystrokes.
  • This custom adversary was then deployed against a newly connected Windows agent (run in a VM).
  • The execution flow showed successful TTPs (directory creation, keystroke logging, PS Tools installation) and failed/blocked TTPs (e.g., Mimikatz due to an outdated script, Powercat waiting for parameters, or files not found to stage).
  • To unblock the powercat TTP, custom facts were injected into the agent: execution.remote.ip (e.g., 192.168.7.1) and execution.remote.port (e.g., 4500). This demonstrated how to provide arbitrary runtime information to agents.
  • Upon re-running, the powercat TTP executed, and a reverse shell connection was successfully established to a netcat listener on the attacker's machine, confirming the dynamic fact injection.
  1. Interactive Shell and Cloud Plugin:
  • The ability to spawn an interactive shell (e.g., PowerShell on Windows, PTY on macOS/Linux) directly from the Operator UI was briefly shown.
  • Finally, the Cloud plugin was demonstrated for provisioning redirectors and Linux servers (with pre-installed Numa agents) in AWS. This showcased how to conduct more realistic, internet-facing campaigns by routing agent traffic through cloud infrastructure, rather than just localhost or internal networks. Multiple redirectors could be spun up and connected, simulating a fleet of C2 infrastructure.

The demo effectively showcased the entire lifecycle of adversary emulation, from CTI analysis to agent deployment, TTP execution, custom adversary building, and advanced C2 capabilities.

Defensive Implications

▶ Watch: Advanced: Deploying Cloud Redirectors for External Operations (97:00)

The core defensive implication of this workshop is the imperative for organizations to adopt a continuous defensive evaluation process. Instead of relying solely on static security assessments or periodic penetration tests, the use of platforms like Prelude Operator enables security teams to constantly test, measure, and improve their defenses.

Key defensive takeaways include:

  1. Iterative Improvement of Detections: By running small, repeatable adversary emulation campaigns, defenders can systematically tune their security controls (e.g., EDR, SIEM rules, network IDS/IPS). Each iteration provides feedback on detection efficacy, allowing for granular adjustments and validation of those changes. This moves beyond a "set it and forget it" mentality to a dynamic, data-driven approach to security.
  2. Operationalizing Threat Intelligence: The ability to translate Cyber Threat Intelligence (CTI) directly into executable TTPs is a powerful defensive tool. Instead of passively consuming threat reports, organizations can actively simulate the behaviors of relevant threat actors (e.g., APT40) against their own infrastructure. This ensures that defenses are aligned with the most pertinent threats.
  3. "Train the Way You Fight": The philosophy of testing in production environments (with appropriate safeguards like the destructive tag) is crucial. While challenging, this approach provides the most realistic assessment of defenses against real-world conditions, including potential blind spots or misconfigurations that might not be apparent in isolated test environments.
  4. Closing the Loop with Lessons Learned: The emphasis on formalizing "lessons learned" is vital. After each emulation, security teams should conduct a "hot wash" to identify:
  • What went wrong (the "big rock" issue).
  • The root cause of the failure (e.g., Windows Defender disabled, lack of SIEM correlation).
  • Specific remedial actions to be taken (e.g., enforcing GPOs, developing new detection rules).

Repeating the emulation after implementing these fixes validates the improvements, creating a tangible cycle of security enhancement.

  1. Granular Detection Capabilities: The platform allows defenders to assess not just whether an entire kill chain was detected, but which specific TTPs within it were caught or missed. This granular visibility helps prioritize defensive efforts and build more robust, multi-layered detection strategies, preventing adversaries from progressing through an attack.
  2. Readiness for Advanced Threats: By emulating complex adversary profiles, including techniques like Powercat reverse shells, credential dumping via Mimikatz, and cloud-based C2 redirectors, organizations can better prepare for and respond to sophisticated attacks that bypass traditional perimeter defenses.

Ultimately, Prelude Operator provides the tools and methodology for security teams to move from theoretical defense to validated, continuously improving security operations, fostering a more resilient posture against evolving cyber threats.

Key Takeaways

  • Continuous Defensive Evaluation is Paramount: Security is not a one-time assessment but an ongoing process requiring constant testing and improvement, driven by small, iterative actions.
  • Operationalize CTI with Executable TTPs: Translate threat intelligence (e.g., from CISA alerts) into concrete, executable Tactics, Techniques, and Procedures (TTPs) mapped to the MITRE ATT&CK framework to validate defenses against real-world adversary behaviors.
  • Prelude Operator Simplifies Adversary Emulation: The platform provides an accessible way to deploy agents (like Numa), execute individual TTPs, and orchestrate complex adversary profiles across Windows, macOS, and Linux systems.
  • Focus on Measurable Improvement: Utilize the platform's logging, variable extraction, and the "lessons learned" framework to identify root causes of defensive failures, implement corrective actions, and then re-test to validate improvements.
  • Leverage Cloud for Realistic Scenarios: The Cloud plugin enables the deployment of redirectors and remote agents in environments like AWS, facilitating more realistic, internet-facing attack simulations that accurately reflect external threat landscapes.
  • Prioritize Repeatability and Automation: The ability to run the same adversary chain repeatedly allows defenders to efficiently test and tune their detection and response capabilities, moving towards a more automated and effective security operations center.

About the Speaker(s)

Alexander Manners is a Principal Security Engineer at Prelude Research, where he leads the security engineering team. His background spans both offensive and defensive cyber operations with the US Air Force. Notably, Alex was one of the lead developers on the MITRE Caldera version 2 platform, working alongside David Hunt, who also now works at Prelude.

Kristopher Willis is a Principal Security Engineer at Prelude Research, where he works as a security researcher. His research interests are diverse, focusing on adversarial AI, automated program analysis (including fuzzing, concolic analysis, and SAT solving), and offensive cyber tool development, such as building the Prelude Operator platform itself. Kris's professional experience includes academia, US Department of Defense contracting (both small and large firms), and work for the Department of Energy national lab space. He also served as a technical lead on AFRL's Black Badge program, which facilitated DARPA's Cyber Grand Challenge (CGC), a competition focused on automated program analysis.

Reviews

Dr. Zero (Offensive Security Researcher) — WEAK

Manners and Willis built MITRE Caldera v2, left, and are now using this workshop slot to demo their startup's tool. It's a feature walkthrough for Prelude Operator, not a research talk. If you need a product demo to learn how to click tabs, fine, but this doesn't advance adversary emulation an inch.

Heather Calloway (CISO) — SOLID

This is a practitioner-level workshop that doubles as product training for Prelude Operator. While your security engineering team will find the hands-on adversary emulation useful, this is not a leadership session. The strategic takeaway, however, is mandatory: if you aren't moving toward continuous control validation, your detection metrics are just guesses.

→ Top-rated talks at Adversary Village @ c0c0n 2021

All talks from Adversary Village @ c0c0n 2021