AI Agents for Exploiting Auth-by-One Errors

AI Agents for Exploiting Auth-by-One Errors

Speakers: Brendan Dolan-Gavitt (AI Researcher, XBOW); Vincent Olesen (AI Researcher, XBOW)

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 3–4, 2026, San Francisco

Watch on YouTube: https://www.youtube.com/watch?v=996zolUsXog

Reading time: ~7 minutes

TL;DR

XBOW researchers Brendan Dolan-Gavitt and Vincent Olesen have built an AI-driven offensive security system that finds and validates authentication and authorization bypasses in web applications — without hallucinating results. The key innovation is "auth transmogrification": an agent-generated script that replays high-privilege requests under a low-privilege context, enabling automated discovery of broken access controls at scale.

Introduction

Authentication and authorization bugs are among the most impactful vulnerabilities in web security, and also among the hardest to find automatically. Unlike memory corruption or SQL injection, these bugs live in logic — in the gap between what a developer intended an access control to do and what it actually enforces. Traditional static analysis tools were never built to reason about intent.

Brendan Dolan-Gavitt, formerly a professor at NYU and now an AI researcher at XBOW, opened the talk by acknowledging the challenge. "These are classically very, very difficult because it's all about verification." At [un]prompted 2026, he and his co-presenter Vincent Olesen explained how they built a system that uses AI agents — paired with rigorous automated validators — to find authentication bypasses, MFA bypasses, and authorization errors in production web applications.

▶ Watch: Why Auth Bugs Are So Hard to Find Automatically (00:00)

The Authentication Problem: Custom Auth Everywhere

The fundamental difficulty begins with diversity. The ecosystem includes HTTP Basic Auth, OAuth 2.0, signed URLs, JWTs, OIDC, OAuth 1.0, Kerberos, magic links, session cookies, TOTP, and LDAP. But as Dolan-Gavitt noted with characteristic humor, "the most common auth mechanism in the world is probably custom auth" — where a developer decided they understood authentication, rolled their own system, and shipped it.

With such variety, there's no hope of writing a deterministic classical validator that can reliably answer yes or no when authentication fails. That's where AI enters. Even when developers do implement a standard like OAuth 2.0, it's often not spec-compliant in practice. The real question is whether the client and backend agree on what authenticated state looks like — not whether the implementation matches any particular specification.

▶ Watch: The Landscape of Authentication Mechanisms (00:00)

The Two-Validator Login Architecture

XBOW's approach to authentication starts with a two-checkpoint system. An agent is given a browser and credentials, then asked to log in. Because agents — like exploits — can produce false positives, the result is passed through two validators before the system trusts it:

1. Login validator: confirms the agent is actually in an authenticated state
2. API validator: confirms the agent has reverse-engineered enough of the application to translate browser state into valid API requests

Only when both validators are satisfied does the system treat the agent as genuinely logged in. The validators themselves were hard-won. As Olesen put it in the Q&A, the team spent significant time building them against "some of the craziest sites you've ever seen, AKA internal banking apps" — edge cases that forced them to iron out corner cases until the validators became truly robust.

▶ Watch: The Two-Validator Architecture (02:00)

From Validators to Offensive Use: Authentication Bypasses

Once the validators exist, they become a powerful offensive primitive. To test for an authentication bypass, XBOW simply removes the credential from the login flow. If the agent still passes both validators — it looks logged in and can make authenticated API requests — then the application has an authentication bypass.

The talk included a live demonstration of this using a real bug previously disclosed by Horizon 3. The application had a default secret key hardcoded in its source code. XBOW's agent found that key, used it to forge session tokens, loaded those tokens into the browser, and achieved admin access — with both validators satisfied and the bypass confirmed.

The same approach applies to multi-factor authentication. To test for an MFA bypass, the team simply removes the second factor from the agent's context during login. If the agent still passes both checkpoints, MFA is broken.

▶ Watch: Authentication Bypass Demo — Hardcoded Secret Key (04:00)

Auth Transmogrification: From Authentication to Authorization

Complete authentication bypasses are relatively rare. The far more common vulnerability class is a working authentication system paired with a broken authorization layer — an application that correctly identifies who you are but fails to properly enforce what you're allowed to do.

This is where Dolan-Gavitt introduced the concept of auth transmogrification, which he called one of the talk's core innovations. The challenge: an admin user browsing an application makes a request to an admin-only endpoint using their bearer token. To test whether that endpoint enforces authorization, you'd need to replay the exact same request under a low-privilege user's context. The problem: a low-privilege user may never see that endpoint in the first place.

Auth transmogrification is a script that an agent generates automatically. Its inputs are a browser state and a URL; its output is a correctly-constructed version of that request under the target authentication context. With this script, the system can make the same request from both privilege levels and compare responses.

XBOW's framework looks for a differential response: the admin gets a user profile back, the low-privilege user gets a 403 Forbidden. That differential is the signal. It tells you the application is attempting an authorization check — but doesn't guarantee the check was done correctly.

▶ Watch: Auth Transmogrification Explained (06:01)

The Attack Agent and Validator-Based Verification

Once a differential response is identified, the system hands off a precise goal to an attack agent: given this endpoint and a low-privilege context, produce the response that the admin would receive. The validator already has the admin response as a ground truth answer key. The attack agent only sees the URL.

If the agent produces the admin response, the authorization bypass is confirmed. If it doesn't, the finding is rejected. This design explicitly solves the hallucination problem: "when you give a capable AI model a task like 'find a security vulnerability no matter what,' it is going to end up making things up," Dolan-Gavitt noted. The validator eliminates that failure mode.

The types of vulnerabilities this framework can detect span a surprisingly wide range:

• JWT forging attacks (altering token claims and re-signing)
• Broken object-level authorization (BOLA/IDOR at the API level)
• Session expiration failures
• Hardcoded tokens
• Parameter bypasses — cases where appending admin=1 to a request grants unexpected access

▶ Watch: The Attack Agent and What It Can Detect (08:01)

Live Demo: Redmine Authorization Bypass

The team demonstrated the system against Redmine, a popular open-source project management tool, where they had previously done extensive manual setup work planting CTF-style flags for verification. The goal of the new approach was to replicate that kind of discovery without the manual setup overhead.

The agent trace showed five iterations from start to solution. The agent got a baseline request and confirmed it could see one project. It then read through source code and — by the fifth iteration — had found the parameter that would bypass visibility checks: admin_projects=1. With that parameter set, the visibility filters are bypassed entirely. The agent confirmed it using the transmogrification script to compare responses with and without the parameter, discovered that a private project was now visible, and submitted the finding to the validator.

The validator's job was straightforward: the agent produced a result that exposed both projects; the admin session also showed both projects; the authorization check was bypassed. Finding confirmed.

Dolan-Gavitt also mentioned a vulnerability that wasn't in the slides: an authorization bypass in a CRM system where changing a single word in a URL from English to French grants access to any document in the system. Because it's not yet fixed, details were shared only under their informal "Bureau Day Vulnerability Program" — buy the researchers a beer.

▶ Watch: Redmine Demo — Finding admin_projects=1 (12:01)

Limitations and Future Directions

The framework has acknowledged scope boundaries. Classical IDORs — where a low-privilege user can access another user's resource by enumerating object IDs — are explicitly excluded. IDORs don't produce the auth-context differential that drives this approach: if I access shopping_cart/5 as one user and another user can also access it, determining whether that's a vulnerability requires judgment about the application's intended behavior.

The system also depends heavily on the quality of the auth transmogrification script the agent generates. However, the approach includes a built-in self-validation mechanism: as agents browse the application, the browser's JavaScript generates requests that get recorded, forming a large automatic test corpus for validating the transmogrification script's correctness.

The Q&A surfaced an interesting future direction: applying this same validator-based approach to testing applications during development, before vulnerabilities ship — using the offensive security system as a pre-production security gate.

▶ Watch: Limitations and Q&A (14:01)

Notable Quotes

"I want you all to write down his name, because a few years from now, you're gonna wanna be able to say 'I saw one of his early talks.'" — Brendan Dolan-Gavitt, introducing Vincent Olesen (00:00)

"When you tell a capable AI model to 'find a security vulnerability no matter what,' it is going to end up making things up. The validator eliminates that false positive problem." — Brendan Dolan-Gavitt (08:01)

"Authentication is actually very hard. And why is it hard? There's so much of it." — Brendan Dolan-Gavitt (00:00)

"This is where we go from evil to diabolical. That's our goal in life — to be as evil and diabolical as possible, for good." — Brendan Dolan-Gavitt, on moving from auth bypasses to authorization exploits (04:00)

Key Takeaways

• Validators are the foundation. AI agents for offensive security only work reliably when paired with deterministic validators that can confirm when a vulnerability has actually been exploited. Without them, the system hallucinates.
• Auth transmogrification enables scale. The ability to automatically generate a script that replays requests across privilege contexts eliminates the need to manually set up test environments for every target.
• Differential responses are the signal. The system targets endpoints where admin and low-privilege users get meaningfully different responses — that differential is both the indicator of an access control attempt and the target for bypass.
• Wide vulnerability class coverage. The same framework catches JWT forging, parameter bypasses, hardcoded tokens, session expiration failures, and broken object-level authorization — not just simple bypasses.
• IDORs require judgment. Classical insecure direct object references are explicitly out of scope because they don't produce the auth-context differential the system relies on.

Slides Reference

Slides are available as 2026-04-04-D1-S1-11-38-AI-Agents-for-Exploiting-Auth-by-One-E.pdf. The slides cover the authentication landscape (HTTP Basic, OAuth, JWT, custom auth), the two-validator login architecture, the auth transmogrification concept, the Redmine demo walkthrough, and the limitations discussion.