Balancing Tradeoffs: Implementing Realistic Adversary TTPs Without Breaking the Bank
Michael C. Long II (Principal Adversary Emulation Engineer · MITRE)
Adversary Village @ c0c0n 2021 · Day 1 · Adversary Village
Overview
In this insightful talk from Adversary Village @ c0c0n, Michael C. Long II, a Principal Adversary Emulation Engineer at MITRE, explores the critical challenge of implementing realistic adversary tactics, techniques, and procedures (TTPs) while navigating the ever-present constraints of limited time and budget. The presentation, aptly titled "Balancing Tradeoffs," delves into practical strategies and real-world examples derived from Long's extensive experience, particularly in leading red team activities for the MITRE ATT&CK Evaluations project. He aims to equip cybersecurity professionals with the knowledge to conduct impactful adversary emulation that truly assesses and enhances an organization's defensive posture, without incurring unsustainable costs or delays.

Key moments
- 0:00 Introduction: Balancing TTPs with time and budget.
- 7:00 Core problem: realistic emulation vs. limited resources.
- 8:30 Understanding quality, time, and cost trade-offs.
- 15:00 Using existing tools: a cost-effective starting point.
- 22:45 Custom tools for unique and sophisticated TTPs.
- 29:00 Driving down costs through TTP reusability.
- 37:30 Automating TTP builds, tests, and execution.
- 42:00 Summary: balancing tradeoffs and emphasizing reusability.
Balancing Tradeoffs: Implementing Realistic Adversary TTPs Without Breaking the Bank
Speakers: Michael C. Long II, Principal Adversary Emulation Engineer, MITRE
Conference: Adversary Village @ c0c0n
YouTube: https://www.youtube.com/watch?v=BRTGyVg3IbQ
Overview
In this insightful talk from Adversary Village @ c0c0n, Michael C. Long II, a Principal Adversary Emulation Engineer at MITRE, explores the critical challenge of implementing realistic adversary tactics, techniques, and procedures (TTPs) while navigating the ever-present constraints of limited time and budget. The presentation, aptly titled "Balancing Tradeoffs," delves into practical strategies and real-world examples derived from Long's extensive experience, particularly in leading red team activities for the MITRE ATT&CK Evaluations project. He aims to equip cybersecurity professionals with the knowledge to conduct impactful adversary emulation that truly assesses and enhances an organization's defensive posture, without incurring unsustainable costs or delays.
Long emphasizes that while the goal is always to achieve the highest fidelity emulation possible, practical considerations often necessitate a strategic approach to TTP development. The talk systematically breaks down what constitutes a TTP implementation, the inherent trade-offs between quality, time, and cost, and various methodologies for achieving a pragmatic balance. From leveraging existing tools to developing custom malware, and crucially, focusing on reusability and automation, Long provides a comprehensive guide for security teams looking to mature their adversary emulation capabilities.
This article will meticulously dissect Long's arguments, outlining the foundational concepts of adversary emulation, the critical problem statement, and the diverse approaches to TTP implementation. It will also highlight the technical nuances, the speaker's expert advice on planning, and the overarching theme of building sustainable, high-value emulation programs that can adapt to evolving threat landscapes and project requirements.
Background
▶ Watch: Introduction: Balancing TTPs with time and budget. (0:00)
Adversary emulation, at its core, involves executing real-world adversary tactics, techniques, and procedures (TTPs) with the primary objective of assessing and improving an organization's cybersecurity defenses. This approach stands in stark contrast to traditional security assessments that might rely on easily bypassed signatures or generic vulnerability scans, which often fail to represent the dynamic and adaptive nature of real-world threats. Long clarifies that TTPs encompass the high-level goals adversaries aim to achieve (tactics), the specific methods they employ (techniques), and the detailed, step-by-step implementations of those methods (procedures). The value of emulating these behaviors lies in its ability to tune defenses against actual adversary actions, rather than against fragile indicators that can quickly become outdated.
The challenge, however, lies in the implementation of these TTPs. As adversary emulation engineers, the desire is to make TTP implementations as representative of real-world threats as possible, striving for an emulation that is "as close to the original TTPs as is practical for our particular project." Long acknowledges that achieving 100% identical replication of observed malware or adversary behavior is often unrealistic and unnecessary; the focus should be on emulating the core behaviors that tend to persist even as adversaries alter their tools or signatures. This pursuit of realism is fundamentally constrained by real-world project limitations: finite time and budget. It's impractical, for instance, to spend an indefinite amount of time recreating every nuance of an APT-29 campaign during a major breach like SolarWinds. The core problem, therefore, is finding the optimal balance between achieving realistic, high-fidelity adversary emulation and delivering these capabilities within defined budgetary and time constraints. This balancing act is ultimately an exercise in managing trade-offs among quality, time, and cost, which forms the central theme of Long's discussion.
Key Findings
▶ Watch: Understanding quality, time, and cost trade-offs. (8:30)
Michael Long’s presentation distills the complex process of adversary TTP implementation into a series of key findings centered on managing inherent trade-offs. The fundamental insight is that effective adversary emulation is an exercise in balancing three core constraints: quality, time, and cost. These three factors are often mutually exclusive, meaning an improvement in one typically comes at the expense of another. For instance, achieving high-quality TTPs quickly will likely incur significant costs, while a low-budget, fast approach will almost certainly compromise quality.
Long defines "quality" in TTP implementation by several characteristics:
- Realism: The emulation must accurately represent real-world adversary behavior.
- Safety: TTPs must be executable in a controlled manner, preventing unintended damage (e.g., ransomware worming out of control).
- Maintainability: Implementations should be reusable across multiple engagements and easily extendable or alterable as project needs evolve.
- Usability: TTPs should be straightforward to build and utilize, even by non-developers, to maximize their operational value.
The talk then explores three primary approaches to TTP implementation, each with distinct implications for quality, time, and cost:
- Using Existing Tools: This involves leveraging native environment utilities ("living off the land binaries" like
tasklistornetstat) or readily available off-the-shelf red team tools such as Mimikatz, Metasploit, or Cobalt Strike. This approach is generally low cost and fast to implement, making it an excellent entry point for new adversary emulation programs. However, its trade-off is often lower quality in terms of unique detection value, as these tools are frequently well-understood and detected by security products. - Using Original Malware: This approach entails using actual adversary malware samples. While it offers maximum authenticity and realism, it comes with significant safety risks and demands a complete, 100% understanding of the malware's capabilities, often requiring extensive reverse engineering. This makes it slow and very high cost, with variable quality depending on the success of safe containment. Long strongly advises against its use in production environments unless under highly controlled, isolated conditions.
- Using Custom Tools: This involves developing bespoke tools based on observed adversary behaviors. It is often the only viable option for emulating highly unique or sophisticated TTPs not found in commercial tools. Custom tools can yield a high return on security investment by forcing defenders to focus analytics on behaviors rather than known tool signatures. However, this method carries high upfront time and cost, requiring specialized developer expertise.
A crucial overarching finding is that reusability is paramount for driving down the long-term time and cost associated with TTP implementation while sustaining quality. Long highlights three pillars of reusable TTPs:
- Clean and Flexible Source Code: Enables adaptation and reuse across different emulation scenarios, significantly reducing future development effort.
- Clear Documentation: Makes TTPs accessible to non-developers, allowing a wider range of staff to execute emulation plans and generate ongoing value.
- Automation: Streamlines builds, unit testing, and execution, ensuring TTPs are consistently production-ready, reducing manual overhead, and enabling continuous assessment.
By strategically planning and prioritizing reusability, organizations can mitigate the initial high costs of custom tool development, transforming them into capital expenses that deliver value over multiple engagements.
Technical Deep Dive
▶ Watch: Custom tools for unique and sophisticated TTPs. (22:45)
Effective adversary emulation, as articulated by Michael Long, necessitates a structured approach to planning and TTP implementation, carefully weighing the trade-offs between quality, time, and cost. Long advises asking three critical questions at the outset of any project: "What specific problems are we trying to solve?", "How much time and budget do I have?", and "How do we measure or define success?". The answers to these questions directly inform the choice of TTP implementation approach.
Approaches to TTP Implementation
- Using Existing Tools:
This is often the recommended starting point for organizations establishing an adversary emulation program. It involves leveraging either living off the land (LOTL) binaries—legitimate system utilities like tasklist, netstat, or ipconfig—or commercially available red team tools such as Mimikatz, Metasploit, and Cobalt Strike. Long refers to a Sophos 2022 threat report graphic illustrating the prevalence of these tools in the wild, noting that while their usage might be trending downward, they still represent significant threats.
- Pros: This approach is low cost and fast because it eliminates the need for new tool development. Security teams can quickly stand up an emulation program, generating immediate value by identifying common detection gaps.
- Cons: The primary trade-off is quality. Many existing tools, especially older versions of Mimikatz or Metasploit's Meterpreter, are often readily detected by mature security products, leading to minimal return on security investment if basic detections are already in place. However, if these prevalent tools do bypass defenses, the return on investment can be substantial, highlighting a critical gap. The challenge is that focusing solely on existing tools might "skew your analysis to the tool rather than the behavior," as security controls might block the known tool itself, rather than the underlying malicious action.
- Using Original Malware:
This approach involves deploying actual malware samples observed in real-world attacks. While offering the highest authenticity in TTP emulation, Long strongly cautions against its general use due to severe safety considerations.
- Challenges: Gaining a complete understanding of original malware capabilities typically requires either access to source code or extensive reverse engineering. Long cites an example of analysts spending 500 hours dissecting over 100,000 lines of Carbonac malware source code. Such an undertaking is immensely time-consuming and costly, and even then, there's a high risk of missing critical functionality that could cause significant damage in a production environment.
- Recommended Use Cases: Long advises restricting original malware use to highly controlled, isolated environments for specific testing, such as assessing security products in a detonation chamber, or for in-depth research. For production networks, the preferred methodology is to analyze the original malware in a lab, understand its low-level TTPs, and then re-implement those behaviors in trusted, custom-developed specimens where the source code is known and controlled. This ensures that only the desired behaviors are emulated, minimizing risk. Consequently, this approach is slow, very high cost, and yields variable quality depending on risk management.
- Using Custom Tools:
This involves developing bespoke tools specifically designed to emulate known adversary behaviors. Long notes that this is often the only viable option for implementing highly unique or sophisticated TTPs that are not supported by existing commercial tools. Examples from the MITRE ATT&CK framework include "adversaries using encrypted virtual file systems stored in the Windows registry" or "hiding bootkit components in unallocated disk space."
- Pros: Custom tools can deliver a great return on security investment because they test defenses against novel or advanced techniques that mainstream red teams might not employ. Crucially, they allow security teams to focus their analytics and detection efforts on the behaviors themselves, rather than relying on signatures for known tools. Since the tools are custom, security controls have no prior biases or analysis, providing a clearer picture of behavior-based detection efficacy.
- Cons: The primary drawbacks are high upfront time and cost. Developing custom tools, especially for advanced TTPs, requires skilled developers and significant effort, making it inherently more expensive and time-consuming than using off-the-shelf solutions.
Driving Down Time and Cost through Reusability
To mitigate the time and cost implications, particularly for custom tool development, Long advocates for implementing TTPs with reusability as a core design principle. This transforms TTP development from an operational expense (one-time use) into a capital expense (long-term value generation).
- Clean and Flexible Source Code:
Maintaining clean, well-structured, and flexible source code is crucial. Long provides an example where his team leveraged existing code from an APT-29 emulation (a traditional RAT) to implement a FIN7 emulation featuring Pillowment malware, a credit card skimmer that iterates through process memory. Despite the distinct TTPs, the underlying code for process iteration (using API calls like CreateToolhelp32Snapshot, Process32First) was reusable, significantly reducing development time for the FIN7 scenario. This demonstrates how a well-maintained code base can serve as a foundation for diverse emulation efforts.
- Clear Documentation:
Thorough documentation is essential for driving down costs by enabling broader usability. This includes documenting source code, build processes, and, most importantly, execution procedures. MITRE's adversary emulation plans and procedures, publicly available on their GitHub, serve as a prime example. These detailed procedures allow non-developers to execute complex adversary emulation plans. By lowering the barrier to entry, more staff can utilize these TTPs, generating additional value and offsetting the initial development costs over time.
- Automation:
Automating various aspects of TTP implementation is a powerful strategy for efficiency and maintainability.
- Automated Builds: Eliminates tedious manual compilation processes. Using tools like
makeorcmakeensures projects build consistently and quickly, freeing up developer time for more critical tasks. It also provides confidence when making code changes, as continuous builds quickly flag regressions. - Automated Unit Tests: For sophisticated TTPs with large codebases (thousands of lines), unit tests are vital. They verify that specific code functionalities work as intended, preventing unintended side effects when changes are introduced and reducing debugging time. This ensures the code remains production-ready and adaptable to changing requirements.
- Automated Execution: MITRE's procedures, often available in YAML format, can be ingested by frameworks like MITRE Caldera for automated TTP execution. This reduces staff overhead during engagements and enables continuous assessment, where TTPs can be run repeatedly as a "leave-behind capability" for network owners. Long addresses concerns about "automating red team out of a job" by emphasizing that human operators remain indispensable for high-risk environments requiring granular control and for ongoing research and development of new TTPs.
By integrating these principles of reusability—clean code, clear documentation, and extensive automation—organizations can build a sustainable and high-quality adversary emulation program that effectively manages time and cost constraints.
Demo / Proof of Concept
▶ Watch: Driving down costs through TTP reusability. (29:00)
While Michael Long's presentation did not include a live demonstration or proof of concept in the traditional sense, he frequently referred to and displayed screenshots of actual TTP implementations and adversary emulation plans developed by his team at MITRE. These artifacts, including detailed procedures and source code examples from emulations like APT-29 and FIN7 (Pillowment malware), are publicly available on MITRE's GitHub repository, specifically within the "C2 Frameworks, Tools, and Utilities" section and the CALDERA project. Long used these real-world examples to illustrate the characteristics of quality TTP implementations, the challenges of working with different types of tools (e.g., Mimikatz, Carbonac source code), and the benefits of reusable code and clear documentation. The visual aids served as concrete evidence of the methodologies discussed, allowing the audience to conceptually understand how these principles are applied in practice without requiring a live execution.
Defensive Implications
▶ Watch: Summary: balancing tradeoffs and emphasizing reusability. (42:00)
Michael Long's discussion on balancing trade-offs in adversary emulation carries significant implications for defenders. The core message for defensive teams is to shift their focus from fragile signatures and tool-specific detections to behavior-based analytics. When red teams are forced to use custom tools or emulate advanced TTPs because existing tools are easily detected, it highlights a critical opportunity for defenders. By moving beyond blocking known tools like Mimikatz (which often "lights up like a Christmas tree" upon execution) and instead focusing on the underlying behaviors – such as process injection, credential dumping techniques, or data exfiltration methods – organizations can build more resilient and adaptive defenses.
The practice of adversary emulation, particularly when leveraging realistic TTPs, serves as an invaluable mechanism for assessing and tuning existing security controls. Defenders can use the insights gained from emulation exercises to identify gaps in their detection capabilities, improve their logging strategies, and refine their security information and event management (SIEM) rules. Long's emphasis on custom tools and advanced TTPs suggests that defenders should invest in capabilities that can detect less common or novel adversary techniques, as these are often the ones that bypass mainstream security products.
Furthermore, the concepts of reusability and automation introduced by Long for red teams also hold immense value for blue teams. By adopting automated TTP execution frameworks like MITRE Caldera, defenders can establish a continuous assessment capability. This allows them to regularly test their security posture against evolving adversary behaviors, ensuring that new detections are effective and that existing controls remain robust. Integrating documented adversary emulation plans into routine security operations can help organizations measure their return on security investment (ROSI) and prioritize defensive enhancements based on actual threat exposure, moving beyond theoretical vulnerabilities to practical resilience against real-world adversaries.
Key Takeaways
- Balancing Trade-offs is Essential: Implementing realistic adversary TTPs requires a strategic balance between quality, time, and cost. These three constraints are often mutually exclusive, necessitating careful planning.
- Prioritize Planning: Before any TTP implementation, define project objectives, understand available time and budget, and establish clear metrics for success. This guides the choice of implementation approach.
- Choose the Right Approach:
- Existing Tools (LOTL binaries, Mimikatz, Metasploit) are cheap and fast, ideal for starting an emulation program, but may offer limited value against mature defenses.
- Original Malware offers maximum authenticity but comes with extreme safety risks, high cost, and requires extensive reverse engineering; best reserved for isolated lab environments.
- Custom Tools are crucial for emulating unique and sophisticated TTPs, driving high return on security investment by testing behavior-based detections, but demand significant upfront time and cost.
- Invest in Reusability: To drive down long-term costs and time, design TTPs for reuse. This involves:
- Clean and Flexible Source Code: Enables adaptation and reuse across different emulation scenarios.
- Clear Documentation: Makes TTPs accessible to non-developers, increasing their operational value.
- Automation: Streamlines builds, unit tests, and execution, ensuring TTPs are production-ready and facilitating continuous assessment.
- Focus on Behavior, Not Just Tools: Defenders should leverage adversary emulation to shift from signature-based detection to understanding and identifying underlying adversary behaviors, which provides more resilient security.
About the Speaker(s)
Michael C. Long II is a Principal Adversary Emulation Engineer at MITRE. In his daily work, he is responsible for emulating adversary tactics, techniques, and procedures with the goal of assessing and improving cybersecurity. He notably leads all red team activities, from development to execution, for the MITRE ATT&CK Evaluations project. Beyond his professional role, Michael is also an academic, currently pursuing a PhD in Cyber Operations with Dakota State University, where he is studying advanced malware analysis. He is actively involved in the cybersecurity community, contributing to various projects and speaking at conferences, including SANS Pen Test HackFest. When not immersed in cybersecurity, he enjoys volunteering for community initiatives, such as tree planting events.
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
A competent but entirely unsurprising methodology talk on managing red team resources. Long knows his trade, but the core thesis boils down to 'write reusable code and don't detonate real malware on production.' No shit.
Heather Calloway (CISO) — STRONG ACCEPT
Michael Long provides a rigorous framework for the economics of adversary emulation. He shifts the conversation from technical mechanics to program management, detailing how to balance fidelity, safety, and cost when testing defenses. If you fund an internal red or purple team, your directors need to be operating on this model.