Building an Autonomous N-day Exploit Generation Pipeline

Building an Autonomous N-day Exploit Generation Pipeline

Speakers: Valentina Palmiotti, Security Researcher and Exploit Developer, IBM X-Force

Conference: BSides Canberra

YouTube: https://www.youtube.com/watch?v=03Xh_6_6U4M

Overview

In this compelling talk at BSides Canberra, Valentina Palmiotti, a distinguished security researcher and exploit developer at IBM X-Force, unveiled BinChomp, an innovative autonomous pipeline designed for generating N-day exploits, specifically targeting the Windows kernel. BinChomp represents a significant leap forward in automated vulnerability exploitation, demonstrating how a blend of traditional deterministic tooling, human-informed workflows, and accessible AI models can dramatically accelerate the exploit development lifecycle. The core motivation behind this research is to challenge the rapidly shrinking window between a patch release and the emergence of functional exploits, making the historical "Patch Tuesday, Exploit Wednesday" a contemporary reality for even independent researchers.

Palmiotti's work directly confronts the evolving landscape of cybersecurity, where the speed of AI-driven exploit generation is rapidly outpacing the average time for organizations to apply critical updates. By meticulously dissecting the exploit development process into manageable stages—some entirely automated by traditional scripts, others guided by AI agents—BinChomp aims to streamline the creation of full-chain exploits from a publicly disclosed CVE. This pipeline not only showcases the formidable capabilities of modern large language models (LLMs) in complex tasks like reverse engineering and exploit crafting but also underscores the enduring value of human expertise in structuring problems and providing targeted feedback to AI.

The talk provides a granular look at BinChomp's architecture, revealing how it gathers artifacts, identifies vulnerability root causes, crafts bug triggers, constructs exploit primitives, and assembles full exploitation chains, all while incorporating iterative verification and a unique "Chompe Cheerleader" feedback mechanism. Palmiotti's research offers critical insights for both offensive and defensive security practitioners, highlighting the urgent need for enhanced patching strategies and a renewed focus on security-by-design principles in an era where sophisticated N-day exploits can be generated with unprecedented speed and cost-efficiency.

Background

▶ Watch: Introducing BinChomp: Autonomous N-day exploit pipeline (0:00)

The concept of "Patch Tuesday, Exploit Wednesday" harks back to an earlier era of cybersecurity, where Microsoft's monthly patch releases were often swiftly followed by the public availability of exploits, sometimes within a day. This rapid turnaround was facilitated by simpler software architectures and fewer mitigations, allowing reverse engineers to quickly identify patched vulnerabilities and weaponize them. However, as operating systems matured and advanced mitigations (such as ASLR, DEP, and Control Flow Guard) became commonplace, the complexity of exploit development surged. Turning a vulnerability into a reliable exploit evolved from a day-long task to one spanning weeks or even months, pushing the "Exploit Wednesday" into a distant past.

This landscape began to shift dramatically around 2022 with the public release of the first GPT models, and more notably, in late 2023 and early 2024, with the emergence of frontier models like Opus 4.6. These models demonstrated an unprecedented capability to assist in vulnerability research and exploit development, even aiding in the discovery and exploitation of zero-day vulnerabilities. Valentina Palmiotti, tracking these advancements, recognized the potential for autonomous exploitation. While initially skeptical about completely autonomous solutions for complex exploits, she observed a significant increase in model capabilities, leading her to develop BinChomp.

BinChomp started as a semi-automated pipeline, designed to augment human productivity by automating tedious tasks. However, anticipating future AI advancements, Palmiotti engineered it for full autonomy, aiming to measure model progress. Coincidentally, during BinChomp's testing phase, Anthropic published an influential N-day study that validated the feasibility of using state-of-the-art models like Mythos to develop full-chain Windows kernel exploits autonomously. This study, which focused on reversing closed-source binaries and generating full chains, provided a crucial benchmark for Palmiotti's independent research.

Palmiotti’s core question was whether an independent researcher, operating on a limited budget and with publicly accessible models, could achieve similar results. Her philosophy, "not everything needs to have AI," guided BinChomp's design. Instead of reinventing every wheel with AI, she leveraged years of exploit development experience and existing deterministic tooling for reliable, repeatable tasks like static analysis, program analysis scripts, and reusable libraries. AI was strategically applied where interpretation, reasoning, and iterative problem-solving were paramount, allowing agents to focus their context on specific problems and interact with tools for deeper investigation. This hybrid approach aimed to achieve competitive results with significantly lower cost and compute resources, bringing "Patch Tuesday, Exploit Tuesday" back within reach for a broader community.

Key Findings

▶ Watch: Overview of the BinChomp pipeline stages (8:10)

BinChomp's experimental results, particularly when compared to Anthropic's N-day study, underscore several transformative findings regarding autonomous exploit generation:

• Competitive Performance with Accessible Models: BinChomp, utilizing the publicly available Opus 4.7 model, achieved remarkable success rates. Across 21 Windows kernel CVEs from January and February 2026 Patch Tuesday updates, BinChomp reproduced 17 bugs and produced 6 verified full-chain exploits. This performance closely rivals Anthropic's state-of-the-art Mythos preview model, which reported 18 reproductions and 8 full chains, and significantly surpasses Anthropic's results with Opus 4.7 (13 reproductions, 0 full chains). This demonstrates that with a well-structured pipeline and human-informed workflows, accessible AI models can yield results comparable to highly restricted, powerful frontier models.
• Drastic Cost Reduction: One of the most striking findings is the cost-efficiency of BinChomp. The entire API bill for all 21 CVE attempts, including successes and failures, was $1,098. This translates to approximately $183 per full chain exploit. In stark contrast, Anthropic reported $15,700 for 8 Mythos chains, averaging around $2,000 per chain. Even when considering bug reproductions, BinChomp's total spend is substantially lower than Anthropic's reported $2,200 for 18 reproductions. This highlights BinChomp's potential to democratize advanced exploit development, making it accessible to researchers with limited budgets.
• Impact of Feedback and Structure: The research clearly showed the critical role of the Chompe Cheerleader agent and exploit chain libraries.
• Chompe Cheerleader: Mandatory consultation with the cheerleader agent significantly improved agent persistence, reduced retry loops, and lowered overall failure rates. It provided both technical guidance and psychological encouragement, helping agents articulate problems and overcome common sticking points in the exploit development process. Without it, agents gave up sooner and failed more often.
• Exploit Chain Libraries: Providing reusable exploit chain logic through libraries shortened development time and reduced retries and failures. This allowed the AI agent to focus on the more specific, structured task of creating exploit primitives from bugs, rather than being overwhelmed by the entire complex task of building a full exploitation chain from scratch. Running the pipeline without both the cheerleader and libraries produced only two full chains, emphasizing their synergistic effect.
• AI's Capability for Complex Bugs: BinChomp demonstrated the ability to produce exploits for complex vulnerabilities, including race conditions and use-after-free bugs. Palmiotti explicitly mentioned that several of the full chains generated involved race conditions, a type of bug notoriously difficult for humans to exploit reliably. This contradicts previous assumptions about AI's limitations in handling intricate timing-dependent vulnerabilities.
• Advancing Model Capabilities: The experiment confirmed that AI model capabilities are rapidly advancing, challenging previous human assumptions about the timeline for autonomous exploit development. Palmiotti herself admitted that her earlier estimate of five years for LLMs to impressively produce exploits independently was incorrect, as "we are there now." This necessitates a fundamental re-evaluation of current security postures.

These key findings collectively demonstrate that autonomous N-day exploit generation is not only possible with publicly available AI models but can also be achieved with high efficiency and significantly reduced cost, provided the process is guided by expert research insights and structured workflows.

Technical Deep Dive

▶ Watch: The 'Chompe Cheerleader' agent for tenacity and feedback (28:57)

BinChomp is architected as a sophisticated, multi-stage pipeline designed for autonomous N-day exploit generation, primarily for the Windows kernel. It strategically combines deterministic tooling with AI-led agents, each responsible for specific tasks within three broad stages: Analyze, Trigger, and Weaponize. Verification stages are interspersed throughout, with failed checks kicking work back to earlier stages for revision.

Pipeline Overview

The pipeline's flow is marked by "peach" for deterministic stages and "coral" for AI-led stages. Green arrows indicate feedback loops where verification failures send the work back for revision. A watcher system monitors each stage for deliverables (reports or runnable programs) or successful verification checks to signal progression.

Stage 1: Gathering Artifacts (Deterministic)

The pipeline initiates by collecting necessary components based on a given CVE.

1. CVE Report Retrieval: The process begins by downloading CVRF data to obtain public information related to the CVE.
2. Binary Identification: A custom mapping associates the vulnerability title with the specific affected binary.
3. Binary and PDB Download: The affected build binaries and their corresponding PDB files (for symbol information) are downloaded. This stage leverages a technique similar to WinBinDex (credited to Michael Maltsev) for reliably locating and retrieving these files from Microsoft's servers. WinBinDex is crucial for obtaining sequential binary versions, eliminating noise in subsequent analysis.

• Output: A set of binary and symbol pairs, organized by build.

Stage 2: Understanding the Change via Feature Flags (Deterministic)

Unlike traditional patch diffing, BinChomp employs a novel approach centered on Windows feature flags.

1. Feature Flag Discovery: Palmiotti's team discovered that Microsoft occasionally ships release builds with internal feature flags that gate vulnerability fixes, sometimes even containing MSRC case numbers. Even when these case numbers are absent, these flags consistently appear, gating patches for vulnerabilities.
2. Focused Analysis: BinChomp compares feature flags between the previous and patched builds, identifying newly added flags. Each flag represents a decision point in the code: one branch (on side) executes the secure, patched behavior, while the other (off side) represents the original, unpatched, vulnerable behavior.
3. Localized Root Cause: Because both code paths (patched and unpatched) are present in the same binary, the analysis is sharply narrowed to the specific conditional code associated with the flag. This eliminates the need for extensive binary diffing, providing a highly focused comparison point for root cause analysis.

• Output: A focused code location and relevant branches associated with the patch.

Stage 3: Tracing Reachability (Deterministic)

Once the vulnerable code location is identified, the next step is to determine how a user-mode program can interact with it.

1. Kernel Entry Points: Vulnerable kernel code is typically reached through defined interfaces such as NT system calls, Win32K system calls, or IOCTL interfaces in kernel drivers.
2. Backward Analysis: Binary Ninja Python scripts are used to perform static analysis, tracing backwards from the patch conditional in the vulnerable code towards these kernel entry points.
3. Path and Conditions: This analysis identifies the specific system calls or IOCTLs that can reach the vulnerable code, along with any input checks or branch conditions encountered along the path.

• Output: Entry analysis, including reachable system calls and parameter conditions.

Stage 4: Root Cause Analysis (AI-led)

This is the first stage where an AI agent takes the lead, interpreting the gathered evidence.

1. Evidence Integration: The agent receives the vulnerable code location, patched and unpatched binaries, and the Binary Ninja entry and parameter condition analysis.
2. Investigation Tools: It can perform further static analysis using tools like Ghidra MCP and Binary Ninja headless.
3. Reasoning and Explanation: The agent's primary task is to explain the root cause of the vulnerability, how the patch addresses it, and how the vulnerability can be exploited. If multiple patches are present, it also correlates the fix to the specific CVE.

• Deliverable: A root cause analysis (RCA) report and a reachability report.

Stage 5: Crafting the Bug Trigger (AI-led)

The AI agent then translates the RCA into a functional program that can reproduce the bug.

1. Program Generation: The agent uses the RCA report and earlier evidence (code branches, entry condition analysis) to write a bug trigger program.
2. Development Environment: It utilizes Visual Studio build tools for compilation and runs the program in a Hyper-V development VM with kernel debugging enabled.
3. Custom WinDbg MCP: A crucial component here is a custom WinDbg MCP (developed with Mike, MMPTE software). This custom wrapper abstracts away the statefulness issues commonly faced by AI agents when interacting with kernel debuggers. It creates a multi-process setup with a long-running KD process and short-lived client connections, logging all debugger output to a file. This allows the agent to perform complex debugging tasks (e.g., conditional breakpoints, inspect register states) without getting stuck.
4. Iterative Cycle: The agent engages in a cycle of writing code, compiling, running, and observing dynamic behavior through the debugger, iterating until a bug trigger is successful.

• Deliverable: An executable proof of concept (PoC) program.

Stage 6: Verifying the Crash (Deterministic)

Before proceeding, the generated PoC is rigorously tested in an isolated environment.

1. Isolated Verification: A PowerShell script powers on a separate, clean Hyper-V VM (distinct from the development VM to prevent agent manipulation).
2. Kernel Crash Check: The candidate PoC is executed, and the system checks for a kernel crash (blue screen).
3. Memory Corruption Analysis: A WinDbg script analyzes the resulting crash dump to confirm that the crash was indeed caused by memory corruption, ensuring it's the desired type of failure.

• Outcome: If verification passes, the pipeline moves to primitive construction; otherwise, it returns to Stage 5 for revision.

Stage 7: Primitive Construction (AI-led)

With a verified bug reproduction, the AI agent now focuses on extracting an exploit primitive.

1. Primitive Definition: An exploit primitive is a specific, controllable capability derived from the bug that can serve as a building block for a full exploit (e.g., arbitrary read/write, increment/decrement, controlled function call, arbitrary free). The agent differentiates between bug classes (e.g., use-after-free) and the resulting primitive's effect.
2. Exploit Helper MCP: This custom MCP packages Palmiotti's extensive research knowledge.

• Object Candidates: For bugs like use-after-free, it provides a list of suitable object candidates that are compatible in size and content for reallocation.
• Function Candidates: For Control Flow Guard (CFG)-related primitives, it suggests function candidates that can be used as "gadgets" when direct ROP is limited, considering parameter control constraints.

1. Backend Logic: The helper uses Binary Ninja headless Python analysis scripts and incorporates logic about allocation compatibility and curated object types known to be exploitable.

• Deliverable: A runnable program that demonstrates the exploit primitive.

Stage 8: Verifying Primitives (Deterministic)

Each claimed exploit primitive is deterministically tested to ensure it behaves as expected.

1. Type-Specific Tests: Every supported primitive type has a predefined, deterministic test.
2. Example: Arbitrary Decrement: For an arbitrary decrement primitive, the test allocates memory, initializes it with a known value (e.g., 7), runs the agent's program (passing the allocated address), and then checks the memory again. If the primitive works, the value should now be 6.
3. Observation Tools: Python coordinates the execution, and WinDbg scripts enable memory observation.

• Outcome: If the observed behavior matches the claim, the primitive passes; otherwise, the implementation returns to Stage 7 for revision.

Stage 9: Exploit Chain Libraries

These libraries are a crucial deterministic component, reusing established exploit logic.

1. Interface-Driven: The agent provides its verified primitives through a defined interface, which includes a callback function (implementing the primitive) and metadata describing its capabilities and constraints (e.g., type, size, number of entries).
2. Windows Exploitation Techniques: The libraries contain pre-built logic for common Windows exploitation techniques. They know how to leverage various primitives (e.g., arbitrary increment, arbitrary free, write-what-where) to construct a full exploitation chain (e.g., privilege escalation from low-privileged user to system).
3. Efficiency and Guardrail Avoidance: This approach prevents the AI agent from having to "reinvent the wheel" for every exploit. It significantly boosts success rates and, importantly, helps avoid AI model guardrails that might trigger when agents attempt to generate "exploity" code like system token overwrites directly.

• Outcome: The library either successfully integrates the primitive into a potential chain or signals insufficient primitives for a complete chain.

Stage 10: Build and Verify Full Chain (AI-led)

The final AI-led stage combines the verified primitives with the exploit chain libraries to produce a complete exploit.

1. Full Chain Generation: The agent uses its verified primitives, the pre-built exploit libraries, and the exploit helper MCP to generate a full-chain exploit.
2. Privilege Escalation: The goal is to escalate privileges from a low-privileged user to system.
3. Final Verification: A verification script starts a clean VM in a low-privileged user context and runs a whoami check. A successful transition to "system" context signifies a verified full chain.

• Deliverable: An executable full-chain exploit. If verification fails, the agent returns to Stage 7 (primitive building) for further iteration.

Chompe Cheerleader Agent (Feedback Loop)

A unique and critical element of BinChomp is the Chompe Cheerleader agent, invoked whenever a verification fails.

1. Hacking Tenacity: Inspired by Palmiotti's own experience in V8 hacking and Pwn2Own, the cheerleader addresses the common problem of AI agents getting "stuck" or becoming pessimistic during complex exploit development.
2. Guided Feedback: The working agent provides a briefing on what went wrong and where it's stuck. The cheerleader, informed by Palmiotti's historical conversations with models, provides both technical advice (e.g., "step through the debugger," "isolate allocator behavior for UAF") and crucial qualitative encouragement ("Remember, hacking involves getting stuck. The best hackers never give up.").
3. Structured Learning: The cheerleader ensures that subsequent retries aren't blind loops but are informed by new understanding, guiding the agent to chip away at the problem. This feedback loop proved instrumental in improving agent persistence and reducing failure rates, often simply by getting the agent to articulate the problem or by providing simple encouragement.

This comprehensive pipeline, with its intelligent division of labor and robust feedback mechanisms, allows BinChomp to autonomously navigate the intricate process of N-day exploit generation, achieving impressive results with accessible AI.

Demo / Proof of Concept

▶ Watch: BinChomp's impressive results and significant cost savings (34:50)

While Valentina Palmiotti's talk did not include a live, real-time demonstration of BinChomp generating an exploit, the entire technical deep dive serves as a comprehensive description of how the pipeline autonomously produces and verifies proofs of concept (PoCs) and full exploitation chains. Within the BinChomp architecture, the "Demo / Proof of Concept" is an output of various stages, not a live presentation component.

Specifically, the pipeline is designed to generate:

• An executable bug trigger proof of concept (.EXE) in the "Crafting the Bug Trigger" stage (Stage 5), which is then verified in "Verifying the Crash" (Stage 6) by ensuring it causes a kernel crash and memory corruption.
• A runnable program demonstrating an exploit primitive in the "Primitive Construction" stage (Stage 7), which is deterministically tested in "Verifying Primitives" (Stage 8) to confirm its claimed behavior (e.g., an arbitrary decrement successfully changing a known memory value).
• A full-chain exploit executable in the "Build and Verify Full Chain" stage (Stage 10), which is verified by demonstrating a privilege escalation from a low-privileged user to the system account in a clean virtual machine.

The talk implicitly demonstrates the capability of BinChomp to produce these artifacts by presenting the successful results of its runs across 21 CVEs, including 17 bug reproductions and 6 verified full-chain exploits. The detailed explanation of the verification stages (Stages 6, 8, and 10) outlines the precise methods used to confirm the functionality and effectiveness of the generated PoCs and exploits within the pipeline's automated workflow.

Defensive Implications

▶ Watch: Q&A: BinChomp's ability to exploit race conditions (46:30)

Valentina Palmiotti's research on BinChomp carries profound defensive implications, fundamentally altering the perceived timeline and cost associated with N-day exploit development. The key takeaways for defenders are stark and urgent:

1. Shrinking Patch-to-Exploit Window: The most critical implication is that the time between a vulnerability patch being released by a vendor and a functional exploit becoming available is dramatically shrinking. The traditional "weeks or months" for complex exploits is now potentially "days" or even "hours" for automated systems like BinChomp. This means organizations can no longer afford to delay patching; immediate deployment of security updates must become a top priority.
2. Democratization of Advanced Exploitation: BinChomp demonstrates that sophisticated N-day exploit generation is no longer exclusive to state-sponsored actors or well-funded research labs with access to proprietary, super-powerful AI models like Mythos. Independent researchers, and by extension, a broader range of malicious actors, can achieve competitive results using publicly accessible and relatively inexpensive AI models (like Opus 4.7) when combined with expert-designed workflows and deterministic tooling. The cost of a full chain exploit has plummeted to as low as $183, making it highly accessible.
3. Increased Urgency for Patch Management: Security teams must re-evaluate their patch management strategies to prioritize speed and efficiency. The luxury of extensive testing cycles for non-critical systems might become a dangerous liability for critical vulnerabilities. Automated patching, robust vulnerability scanning, and continuous monitoring for newly released patches are more crucial than ever.
4. Emphasis on Security by Design and Architecture: As software development accelerates ("everybody's vibe coding their little hearts out"), the importance of security by design principles cannot be overstated. Proactive security considerations during the software development lifecycle will significantly reduce the number and severity of vulnerabilities. Retroactively fixing security flaws after deployment is a far more costly and headache-inducing endeavor in an era of rapid exploit generation. Investing in secure coding practices, threat modeling, and architectural reviews from the outset can mitigate the impact of N-day vulnerabilities before they even emerge.
5. Beyond Memory Corruption: While BinChomp is currently optimized for memory corruption bugs, the underlying principles of autonomous analysis and exploit generation are generalizable. Defenders should anticipate AI pipelines evolving to tackle a broader spectrum of vulnerability types, including logic bugs, race conditions (which BinChomp already handles), and other complex flaws.
6. Rethinking Adversary Simulation: For organizations engaged in adversary simulation, the capabilities demonstrated by BinChomp mean that simulated adversaries can now leverage highly effective, autonomously generated N-day exploits. Red teams and penetration testers can use similar methodologies to enhance their capabilities, but blue teams must also prepare for adversaries who do not require extensive manual effort to weaponize known vulnerabilities. This necessitates a focus on EDR evasion and stealth, which BinChomp currently doesn't prioritize but represents a future area of development for automated systems.

In essence, BinChomp serves as a potent wake-up call, signaling a new era where the speed and cost of N-day exploit development have been drastically reduced. Defenders must adapt by accelerating their patching processes, prioritizing security at every stage of software development, and challenging their own assumptions about the capabilities of adversaries in an AI-driven world.

Key Takeaways

• AI Models are Rapidly Advancing: Modern large language models (LLMs) like Opus 4.7 are incredibly powerful and their capabilities in vulnerability research and exploit development are progressing at an unprecedented rate, far exceeding previous human estimations.
• Human Expertise and Structured Workflows Remain Critical: While AI is powerful, combining human research insight, pre-existing deterministic tooling, and well-structured workflows allows accessible AI models to achieve competitive results with significantly lower costs and compute resources than state-of-the-art frontier models.
• Exploit Development is Faster and Cheaper: The time and financial cost of developing N-day exploits are drastically decreasing. BinChomp demonstrated full-chain exploits for Windows kernel vulnerabilities at approximately $183 per chain, making advanced exploitation accessible to a wider range of actors.
• Shrinking Patch-to-Exploit Window: Organizations must prioritize immediate patching and rapid deployment of security updates. The window between a vulnerability fix being released and a functional exploit becoming available is now measured in days, not weeks or months.
• Security by Design is Paramount: With the accelerated pace of exploit generation, focusing on security by design and robust security architecture from the outset of software development is more crucial than ever to minimize vulnerabilities and reduce the burden of reactive security fixes.
• Challenge Your Assumptions: The rapid evolution of AI capabilities demands that security professionals continuously challenge their assumptions about what's possible, adjust their strategies, and adapt to the evolving threat landscape where autonomous N-day exploit generation is a present reality.

About the Speaker(s)

Valentina Palmiotti, who also goes by her handle Chompe, is a highly respected Security Researcher and Exploit Developer. She is a prominent figure at IBM X-Force, where she leads the X-War vulnerability research team. Palmiotti describes herself as a "weird machine mechanic," dedicating much of her time to investigating the intricate and often unexpected behaviors of complex systems. She is known for her expertise in exploit development, including V8 hacking and participation in prestigious competitions like Pwn2Own. Valentina Palmiotti was previously a keynote speaker at BSides Canberra, underscoring her standing and influence within the security community. Her work consistently pushes the boundaries of offensive security research, providing critical insights into the future of vulnerability exploitation.