Promp2Pwn – LLMs Winning at Pwn2Own

Promp2Pwn: How an LLM Agent Found the Bugs That Won Pwn2Own

Speaker: Georgi G (Georgi Geshev), Director of Research, Interrupt Labs

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 4, 2026, San Francisco

Watch on YouTube: https://www.youtube.com/watch?v=c5XAvRbma6Y

Reading time: ~10 minutes

TL;DR: Georgi Geshev of Interrupt Labs built an agentic AI system to hunt Android vulnerabilities for Pwn2Own — and it worked. The system found over a dozen bugs in Samsung pre-installed apps; two were chained into a successful contest entry that silently hijacked a phone's camera feed with a single link click. The path from naive automation to real results was an education in treating the LLM like an intern, not a god.

Ten Years of Pwn2Own and Getting Bored

▶ Watch: Introduction and motivation for AI (0:00)

Georgi Geshev has competed in Pwn2Own for over a decade, focused primarily on Android targets. By his own account, he was getting bored. "Nothing had really changed for a couple of the vendors I'd been targeting. It was essentially the same bugs over and over again."

Finding those bugs was getting harder, not easier — they were buried across a huge number of applications, and bug bounty programs were eliminating the obvious low-hanging fruit. That created what Geshev called "a great excuse to teach myself AI."

The rationale for applying LLMs specifically to Android vulnerability research was sound:

• High-fidelity decompilation output. Android's Dalvik executable format decompiles cleanly to human-readable Java — excellent input for a language model.
• Pattern-matching workload. Many Android vulnerabilities are pattern-recognizable: when you see one, you know instantly something's wrong. Static analyzers often miss them; an LLM is better suited to this kind of implicit pattern matching.
• No privacy concerns. In day-to-day professional vulnerability research, LLMs are difficult to use because of sensitive client data. Pwn2Own is different — these bugs are going to the contest organizers regardless.

Before committing to an agentic approach, Geshev tested the premise manually: copy-pasting decompiler output from JEB or JADX into a chatbot. "Surprisingly, some bugs got picked up instantly. There was clearly something there."

Why Existing Tooling Wasn't Enough

▶ Watch: AI-assisted Android RE — the gap in tooling (2:00)

Geshev experimented with Android Studio — decompile with JADX, export as a Gradle project, import into Android Studio, and use JetBrains AI Assistant or GitHub Copilot as assistants. The integration was disappointing. Plugins were focused on writing code, not analyzing it, and refactoring support was poor.

In August 2025, he looked for agent-based tools that could simply consume an Android application and go find bugs. Nothing existed. "That was another good excuse to try and learn something new."

The Initial Architecture

▶ Watch: High-level design and the JADX MCP (6:00)

The system Geshev built uses:

• LangChain for the agent framework
• JADX to decompile Android applications to Java
• LiteLLM for model management
• Azure Foundry for model hosting
• LangFuse for introspection and observability

The initial design was a two-agent setup:

1. Attack surface analysis agent — parses the AndroidManifest.xml, identifies entry points, and produces a structured list
2. Bug hunter agent — takes that list and analyzes each component for vulnerabilities

Both agents communicate with JADX via a JADX MCP server — a JADX plugin that runs an embedded Javalin HTTP server, exposes JADX's internal API, and wraps it with a FastMCP Python layer. The MCP tools include:

• get_android_manifest — retrieve the full app manifest
• get_method_source — source code for a specific method
• get_class_source — source code for a full class
• find_method — search for methods by name across all classes

This MCP implementation was based on Jafar Pathan's open source work (Apache 2.0). Geshev acknowledged it wasn't ideal but was the only option available at the time.

The Early Mistakes: "Find Bugs. This Is the App. Go."

▶ Watch: Early mistakes and vague objectives (8:00)

The first attempts were humbling. The initial prompt was essentially: "Looking for bugs. This is the app. Go find bugs." The result was an agent that got lost, ran out of tokens, hit tool call limits, and produced nothing useful.

"The objective was extremely vague. The success criteria were completely fuzzy. The agent didn't know what kinds of bugs to look for or exactly where to look."

After getting feedback that he was "absolutely misusing this," Geshev made a series of focused improvements:

Fix 1: Structure the objective. Instead of "find bugs in the app," the prompt became "find bugs in this one particular component — specifically, a component reachable through this intent filter for this data format." That gave the agent a specific entry point, a narrower call graph to traverse, and meaningful context to stay on track.

Fix 2: Fix the MCP tooling. Tool descriptions were vague. Code search was extremely slow. Error messages were opaque — whenever a tool call failed, it returned "there's been an error" with no further context. Actionable error messages matter enormously: they let the agent understand what went wrong and retry with adjusted inputs.

Fix 3: Evolve the system prompt. This was the most important change. "I realized I was dealing with an intern. If I had an intern for the summer and I wanted them to find bugs for Pwn2Own, I'd need to be very explicit about the process I have deeply ingrained in my brain."

The final system prompt included:

• An explicit attacker model and the primitive chains needed for a successful exploit
• The assumption that a single user interaction (e.g., one deeplink tap) is the entry point
• MITM conditions for cleartext HTTP traffic
• A specific set of compromised domains to assume control over
• Explicit bug class priorities (WebViews, file operations)
• "Do not suggest fixes or mitigations!!!" — the agent kept wasting tokens on remediation advice nobody asked for

▶ Watch: System prompt evolution (10:00)

The Pwn2Own Entry: Two Bugs, One Chain

▶ Watch: The Pwn2Own bugs (12:00)

The agent — still basic, minimal null verification, little guardrailing — found over a dozen bugs that required manual verification. Two were chained together into the Pwn2Own Samsung entry.

Bug 1: Smart Touch Call — Broken URL Validation

Smart Touch Call is Samsung's interactive customer service application. The vulnerability is a classic broken URL validation problem.

The entry point is WebViewActivity, defined in the AndroidManifest with scheme stc.scheme and host stc, marked as exported. When a user (or attacker) sends a URI matching this pattern, the app extracts several intent extras: URL, CRID, hashCode, httpMethod.

The handleIntent() method applies a regex check when httpMethod equals "POST": it verifies the URL starts with storedUrl — which returns an empty string. That means any URL passes validation. The code calls webview.postUrl(baseUrl, ...) with an attacker-controlled URL.

The WebView has a custom WebChromeClient implementation that silently grants the WebView access to the microphone, camera, geolocation, and other capabilities — no notification, no permission prompt, no per-origin confirmation. Executing JavaScript in this WebView gives the attacker silent access to all of them.

The agent's bug report correctly identified the data flow and the sink (WebView.postUrl()), flagged JavaScript execution, and provided enough evidence to reproduce the issue. What it missed: the custom WebChromeClient implementation that grants the capabilities, and a critical prerequisite — the app only processes the intent if an active phone call is ongoing. "That's a very strange prerequisite for an exploit, but it's also what made it the perfect meme exploit for the contest."

Bug 2: Bixby — Permissive Domain Validation

The agent also found a vulnerability in Bixby, Samsung's AI assistant. "Which is both nice and ironic."

The Bixby deep-link handler accepts any subdomain of mcsvc.samsung.com as a valid load target in its WebView. The domain space is enormous — many subdomains and sub-subdomains exist, and some of them carry cross-site scripting vulnerabilities. Loading one of those XSS-vulnerable Samsung subdomains into the Bixby WebView executes the attacker's JavaScript at that origin, which can then reach a highly privileged JavaScript interface exposed within the Bixby WebView — providing access to Bixby components that are not normally accessible to other applications on the device.

The agent's bug report correctly flagged the permissive host validation, suggested looking for existing XSS on broad Samsung domains, and noted the privileged JavaScript interface.

▶ Watch: The Pwn2Own demo — one link, camera access (18:01)

The Chain

The exploit chain at the contest demonstration: victim phone on the left, attacker clicks one link.

1. The deeplink triggers the Bixby WebView via an XSS-vulnerable Samsung subdomain
2. JavaScript in the Bixby WebView instructs Bixby to make a phone call
3. The active phone call satisfies Smart Touch Call's prerequisite
4. The attacker now loads Smart Touch Call's WebView with an attacker-controlled URL
5. Silent access to the camera feed is achieved

One link click. No permissions dialogs. One successful Pwn2Own entry.

The exploit chain required human creativity to construct — the agent could not reason about how two bugs across two applications could be combined. "Human creativity was required to construct the exploit chain."

Agent Evolution After the Contest

▶ Watch: Agent improvements and scale (18:01)

After the Pwn2Own entry, the system went through several rounds of improvement:

APK harvesting. A set of scripts to retrieve APKs from app stores and start scanning at scale — moving from one app at a time to bulk analysis.

Decoupled manifest analysis. The attack surface analysis agent's AndroidManifest parsing was extracted into a standalone deterministic script. "I realized there's no point using an LLM for something entirely predictable where I know exactly what I'm looking for." This is a key principle: LLMs for pattern recognition, deterministic code for deterministic tasks.

Refined entry point scoping. Intent filters often contain multiple distinct paths — sub-entry points — that are worth treating as unique entry points. This change significantly increased the agent's depth of analysis per component.

Inline de-obfuscation. Some apps are lightly obfuscated. The agent works much better on de-obfuscated code. A de-obfuscator agent was added inline: whenever the bug hunter requests source code, the request goes through the de-obfuscator, which calls the JADX MCP, fetches the source, replaces one-to-three letter method and class names with meaningful identifiers, and returns the cleaned code. The improvement in agent performance was substantial.

▶ Watch: Multi-run aggregation and the bug verifier (20:01)

Multi-run aggregation. Running the agent multiple times on the same entry point produces different results — inherent to LLM non-determinism. Geshev now runs the agent one to five times per entry point depending on interest level, aggregates the bug reports, and runs them through a deduplication step.

Bug verifier agent. A separate agent that reviews bug hunter reports and flags false positives. It correctly identified that the Smart Touch Call bug was only reachable during an active phone call — which is why the Pwn2Own chain needed Bug 2. "It's a double-edged sword, because if the agent decided this is not reachable, I would have never seen it probably."

Jinja2 (Ginger) for prompt management. With multiple agents sharing system prompt components, Ginger (Jinja2) was used to manage shared prompt snippets across the agent set.

Sootup for call graph analysis. A source-to-sink reachability tool based on Sootup — a Java bytecode processing framework — was added to analyze paths through the call graph and validate whether a bug hunter's reported data flow is actually reachable.

What's Next

▶ Watch: Future roadmap (24:02)

Geshev outlined the next planned improvements:

• Static analyzer integration: Run existing static analysis tools and let the agent triage their output rather than replacing them
• Cross-app bug tracking: A database where the agent can see bugs across multiple applications simultaneously, enabling it to reason about chains without human prompting
• PoC generation and runtime validation: Plug in a phone via ADB MCP and Frida MCP, let the agent test the trigger produced by the bug hunter, and refine the proof-of-concept while collecting runtime events

Notable Quotes

"I realized I was dealing with an intern. If I had an intern for the summer and I wanted them to find bugs for Pwn2Own, I'd need to be very explicit about the process I have deeply ingrained in my brain." (10:00)

"Do not suggest fixes or mitigations!!!" (from the system prompt)

"There's no point using an LLM for something entirely predictable where I know exactly what I'm looking for." (on decoupling deterministic analysis)

"Human creativity was required to construct the exploit chain." (12:00)

Key Takeaways

• Android's clean Java decompilation output makes it an unusually good target for LLM-based vulnerability research — the input format is well-suited to pattern recognition.
• Vague agent objectives produce useless results; telling an agent "find bugs in this app" is like telling a new hire "make the product better." Scope must be explicit: specific component, specific intent filter, specific bug class.
• LLMs should handle pattern recognition and structured hypothesis generation; deterministic code should handle deterministic analysis (manifest parsing, call graph reachability).
• Inline de-obfuscation — cleaning up one-to-three letter identifiers before the bug hunter sees the code — makes a significant difference to agent output quality.
• The current limitation is cross-app reasoning: agents see bugs within one application but cannot yet reason about multi-app exploit chains without human direction.

Slides reference: Slides PDF available (2026-04-04-D2-S1-10-51-Promp2Pwn-LLMs-Winning-at-Pwn2Own-pdf.pdf). Key slides covered the JADX MCP architecture, the Smart Touch Call entry point code (WebViewActivity manifest entry and handleIntent() source), the bug hunter report, and the Bixby WebView URI path validation code. Transcript PDF (2026-04-04-Day2-Stage1-G-txt.pdf) provided detailed narrative content.