Automating Your Job? The Future of AI and Exploit Development

Automating Your Job? The Future of AI and Exploit Development

Speaker: Perri Adams (DARPA)

Conference: OffensiveCon 2025 — May 16–17, 2025, Berlin

YouTube: https://www.youtube.com/watch?v=Y1naY3gupRw

Reading time: ~12 minutes

TL;DR

Perri Adams, former DARPA program manager who launched the AI Cyber Challenge, presents a technically grounded assessment of frontier AI models' (GPT-4o, o3, Claude) actual capabilities in exploit engineering, using the regreSSHion double-free in OpenSSH 9.1 as a live case study. The central argument is that the gap between identifying a vulnerability and producing a working exploit — navigated through heap grooming, leak primitive development, and ASLR bypass against modern mitigations — remains far wider than current AI hype acknowledges, and that policy discourse needs to catch up with technical reality.

Introduction

The opening keynote of OffensiveCon 2025 arrived at a moment when AI companies were making increasingly confident claims about autonomous vulnerability discovery and exploit generation. Perri Adams, who directed DARPA programs on automated cyber reasoning systems and launched the AI Cyber Challenge at DEF CON — a competition focused specifically on AI for vulnerability discovery — chose to engage those claims head-on with a live technical case study rather than management-layer talking points.

The distinction Adams drew from the outset is critical: AI for vulnerability discovery and AI for exploit generation are different problems of vastly different difficulty. Current AI research has made genuine progress on the former. The latter requires navigating the operational complexity of modern memory-safety mitigations — ASLR, stack canaries, glibc tcache hardening, PIE — across a chain of subtasks (heap grooming, leak primitive construction, control flow hijack) where each step depends on precise environmental conditions. The talk examines whether frontier models can actually contribute to that chain, using a real pre-authentication double-free in OpenSSH 9.1 as the test case throughout.

The regreSSHion Double-Free: Bug Mechanics

The vulnerability at the center of the talk is a double-free in OpenSSH 9.1 discovered by Qualys and reported to the oss-security mailing list in February 2023. The bug is triggered by a specific interaction during the SSH handshake: a char pointer cp is freed during the processing of a legacy SSH_OLD_DH_GEX key exchange message — the type of message emitted by a Windows 98 SSH client — and then freed again along a separate code path when matchfilter_denylist is invoked. The triggering condition requires a client to send the legacy SSH2_MSG_KEX_DH_GEX_REQUEST_OLD message type, which no modern SSH client sends by default.

The sshd architecture compounds the exploitation challenge. When a client connects, the server immediately forks two processes: an unprivileged child handling most handshake logic, and a separate privileged monitor running as root. They communicate via an IPC pipe. The unprivileged child processes the key exchange, so the double-free occurs in an unprivileged context — getting code execution in the root monitor requires a working exploit in the child, which then needs to leverage the IPC channel to escalate. Modern mitigations active in OpenSSH on Ubuntu/Debian — position-independent executables, full ASLR, and stack canaries — make each step non-trivial.

Qualys initially assessed the bug as not practically exploitable on Linux due to glibc's fine-grained tcache bucketing. Adams disagreed with that assessment, noting that tcache only holds seven free chunks per size class before subsequent frees are directed to the unsorted bin, which has different double-free protections and supports chunk coalescing. OpenSSH 9.1 running against glibc 2.37 — which uses exactly this seven-chunk tcache limit — was the target environment Adams chose to investigate.

▶ Watch: regreSSHion bug mechanics and architecture (4:00)

Heap Grooming: Forcing the Chunk into the Unsorted Bin

The first exploit sub-task is coercing the double-free chunk out of the tcache (where protections make the double-free non-exploitable) and into the unsorted bin (where coalescing is possible). This requires pre-filling the relevant tcache bin before the vulnerable code path executes.

Adams's analysis of the pre-authentication SSH protocol identified the only attacker-controlled heap allocations that occur before the key exchange function is called: the algorithm negotiation lists. During the initial SSH handshake, both client and server exchange lists of supported key exchange algorithms, MAC algorithms, public key algorithms, and compression algorithms. The client controls the size of these lists, and because the lists are sent twice in the negotiation flow, an attacker can allocate a predictable sequence of heap chunks of controlled sizes. By crafting algorithm list strings of the right lengths, the attacker fills the tcache bin corresponding to the double-free chunk's size class with seven entries, so that when the bug is triggered, the freed pointer lands in the unsorted bin.

The practical obstacle was not the heap grooming logic itself — it was implementing it. No production SSH client accepts arbitrary algorithm negotiation strings or willingly violates the RFC in the ways needed. Adams resolved this by dismantling the Paramiko Python SSH library and constructing a custom Frankenstein client capable of sending the non-standard pre-auth messages. This custom client became the exploit skeleton for all subsequent heap manipulation.

▶ Watch: Algorithm list grooming and custom SSH client (12:00)

Evaluating AI Models on Exploit Sub-Tasks

With the exploit skeleton established, Adams used the regreSSHion case as a structured benchmark for evaluating frontier AI models — GPT-4o, o3, and Claude — on specific exploit engineering sub-tasks. The methodology involved presenting each model with the vulnerable code, the relevant glibc heap allocator behavior, and the exploit context, then asking for concrete steps to advance the exploit.

The results were nuanced. Models demonstrated genuine usefulness at the level of code summarization, explaining what a function does, identifying which libc functions were being called, and producing readable prose descriptions of heap allocator behavior. They were less useful — and sometimes confidently wrong — when asked to reason about multi-step interactions between the SSH protocol state machine, the heap layout, and the tcache internals. Prompting models to suggest heap grooming strategies produced generic observations about tcache and unsorted bin behavior that were technically accurate but lacked the operational specificity needed to construct working sequences of protocol messages.

More critically, Adams found that the models struggled with the dependency chain inherent to exploit development: each sub-task's inputs depend on the outputs of the previous one, and errors compound. A model that correctly describes unsorted bin coalescing in the abstract may nonetheless suggest a grooming sequence incompatible with the actual allocation pattern of a specific OpenSSH build against a specific glibc version. Developing a working exploit requires iterative experimentation with a live target — something models cannot do without tool-augmented feedback loops.

▶ Watch: AI model evaluation on heap grooming tasks (16:00)

The Mitigation Gap and What It Means for AI

The talk's most important structural argument concerns the asymmetry between vulnerability identification and exploit development. Finding that a double-free exists in sshd's pre-auth code is a tractable pattern-matching problem: identify a pointer freed on one code path and freed again on another. Modern static analysis tools, fuzzing infrastructure, and even LLM-assisted code review can contribute meaningfully to that search.

Converting that finding into a working remote code execution exploit requires solving the following interdependent problems against a hardened target: heap layout control (constrained to pre-authentication protocol messages), leak primitive development (no format string, no ASLR bypass obvious in pre-auth code), ROP chain construction (PIE binary, gadgets must be computed relative to leaked base addresses), and reliable triggering (the exploit must work across multiple connections without crashing the server in a way that kills the fork). Modern mitigations do not make this impossible — Adams notes that the Qualys team demonstrated instruction-pointer control on OpenBSD — but they dramatically raise the bar and make the problem environment-specific in ways that constrain broad applicability.

This gap, Adams argues, is precisely why framing AI as an imminent autonomous exploit generator overstates the technology's current capabilities and misleads policy discussions. AI tools provide valuable leverage at specific points in the exploit development workflow — code comprehension, documentation search, initial grooming strategy brainstorming — but they do not yet automate the closed-loop, environment-specific reasoning required to produce reliable exploits against patched, mitigated targets.

Policy Implications and the AI Cyber Challenge

Adams framed the policy discussion with reference to DARPA's AI Cyber Challenge, which she launched at DEF CON, a competition specifically focused on AI for vulnerability discovery rather than exploit generation. Even in the more tractable vulnerability discovery domain — where the problem is closer to a structured code analysis task — the challenge revealed substantial gaps between model capabilities and human expert performance on real-world codebases. The distinction matters for export controls, offensive capability attribution, and defensive investment prioritization: a world where AI can reliably discover bugs in open-source software requires a different policy response than a world where AI can generate working remote exploits for deployed network services.

The talk concluded with a call for technical practitioners — the OffensiveCon audience specifically — to engage directly with policy processes, bringing concrete empirical assessments of AI capabilities to forums where decisions about offensive cyber capability governance are being made. Advocates for strong AI restrictions and advocates for minimal restrictions both tend to reason from the vulnerability discovery use case; the exploit development case is harder and should be center stage.

Notable Quotes

"What they don't tell you when you become a senior manager or director is that you're gonna get lobotomized. Someone's gonna put you under and cut out that part of your brain that has the technical bits in it."

— Perri Adams, ▶ 4:00

"glibc does take a more fine-grained approach to bucketing heap chunks, but given enough flexibility, you can groom the heap appropriately. tcache only takes the first seven chunks — after that, it'll get forced into something like the unsorted bin, which has different double-free protections but also allows you to coalesce chunks."

— Perri Adams, ▶ 8:00

"They have modern mitigations that, to be frank, are the reason why this is so hard, if not nearly impossible to exploit. At a time where there is an ongoing debate about the efficacy of mitigations, I think that's something to note."

— Perri Adams, ▶ 10:00

Key Takeaways

• Vulnerability discovery ≠ exploit development: AI models have shown genuine progress on the former (pattern recognition in source code), but the latter requires closed-loop, environment-specific reasoning across a chain of interdependent sub-tasks that current models do not reliably perform.
• Modern mitigations matter: The regreSSHion case study demonstrates that stack canaries, ASLR, PIE, and glibc tcache hardening collectively transform a double-free in high-value network software from a straightforward primitive into a research problem — the gap between "bug identified" and "working RCE" is not a footnote.
• Heap grooming is constrained by protocol surface: For pre-authentication vulnerabilities in network daemons, the only attacker-controlled allocations before exploitation are those the protocol itself permits; identifying and leveraging those allocations (e.g., SSH algorithm negotiation lists) requires deep protocol knowledge that generic AI models lack.
• Custom client tooling is a prerequisite: No production SSH client will violate the RFC in the ways needed for exploit-level heap grooming; building a custom Paramiko-derived client was necessary to perform even the first step of the exploit chain.
• Technical practitioners must engage policy discourse: Policy discussions about AI and offensive cyber capability are happening now and are frequently poorly grounded in technical reality; the exploit development community has unique expertise to contribute and an interest in ensuring that governance decisions reflect actual capability assessments.