Q&A - AI's Journey Through Zero-Days And A Thousand Bugs

AI's Journey Through Zero-Days And A Thousand Bugs

Speakers: Jo (Primary Speaker), Brendan (Validator Specialist), Albert (Head of AI, Expo), Vincent (AI Researcher, Expo)

Conference: Bug Bounty Village

YouTube: https://www.youtube.com/watch?v=0rh7fRXphJs

Overview

This article delves into the groundbreaking work presented at Bug Bounty Village, where a team of researchers from Expo showcased their advanced autonomous pentest AI. While originally scheduled as a full presentation, the session evolved into an engaging Q&A, providing attendees with an intimate look into the intricacies of building an AI capable of discovering and validating a wide array of vulnerabilities. The core of their innovation lies in creating a system that not only hunts for bugs but also rigorously validates them, effectively eliminating the pervasive issue of false positives and "hallucinations" often associated with AI-driven security tools.

The talk, primarily led by Jo, illuminated the architecture and operational methodologies behind this sophisticated AI, emphasizing its ability to mimic human pentesting workflows while maintaining strict ethical and scope boundaries. This technology represents a significant leap forward in automated security testing, offering the potential to dramatically scale vulnerability discovery, enhance efficiency, and provide continuous, comprehensive security assessments. For security professionals, bug bounty hunters, and developers, understanding this AI's capabilities and limitations is crucial as it reshapes the landscape of proactive cybersecurity.

Background

▶ Watch: Autonomous pentest AI architecture overview (0:00)

The cybersecurity industry has long grappled with the challenge of efficiently identifying vulnerabilities across ever-expanding digital landscapes. Traditional manual penetration testing, while highly effective, is resource-intensive, time-consuming, and difficult to scale. Automated scanning tools have existed for decades, but they often struggle with context, complex logic, and the high rate of false positives, requiring significant human oversight to filter actionable findings from noise. This bottleneck has spurred a quest for more intelligent automation, particularly with the advent of advanced Artificial Intelligence and Large Language Models (LLMs).

However, integrating LLMs into security testing introduces its own set of problems. LLMs are known for "hallucinations"—generating plausible but incorrect information—and a tendency to "cheat" or find shortcuts to satisfy a prompt rather than genuinely solve a problem. This makes them inherently unreliable for tasks requiring high precision, such as vulnerability validation. Prior work in automated bug hunting often focused on specific vulnerability classes or relied on symbolic execution and static analysis, which can miss runtime issues or complex logical flaws. The Expo team’s work addresses these fundamental challenges by building a comprehensive, multi-component AI system designed not just to find vulnerabilities but to autonomously validate them with a high degree of confidence, bridging the gap between raw AI capability and practical, reliable security outcomes. Their system aims to replicate the strategic thinking of a human pentester, from reconnaissance and task prioritization to intelligent exploitation and robust verification, all while adhering to defined scope and ethical guidelines.

Key Findings

▶ Watch: Wide range of supported vulnerability classes (4:00)

The Expo team's presentation, despite its Q&A format, revealed several pivotal findings and contributions in the realm of AI-driven security testing:

1. Autonomous Pentest AI Architecture: The successful development and deployment of a multi-component AI system capable of end-to-end blackbox penetration testing. This architecture, comprising a Coordinator, Agents, Validators, and a Proxy, demonstrates a scalable and intelligent approach to vulnerability discovery.
2. Robust False Positive Elimination with Validators: A critical breakthrough is the Validator component, which effectively removes false positives and mitigates LLM hallucinations and "cheating." This component, as detailed in Brendan's Black Hat talk, uses techniques like headless browsers (e.g., Puppeteer) to objectively confirm vulnerability exploitation (e.g., verifying an XSS pop-up), achieving near-zero false positive rates for many vulnerability types.
3. The "Alloy Models" Concept for Enhanced Performance: The introduction of Alloy Models represents a novel approach to leveraging multiple LLMs. By dynamically selecting between different models for queries—often a "flip of a coin" between two distinct models—and meticulously managing context, the team discovered that combinations of diverse models consistently outperformed individual models. This "alloying" effect is particularly beneficial for broader vulnerability discovery objectives, as different models may excel in different areas (e.g., one for XSS, another for application crawling).
4. Optimized Resource Utilization and Efficiency: Through extensive testing and data analysis, the team identified the "diminishing returns" principle in AI-driven vulnerability scanning. They learned that "more runs doesn't mean better results" and optimized the number of runs and agents based on vulnerability type. For instance, exposed secrets might require fewer runs than complex RCE vulnerabilities, leading to significantly more findings with fewer attempts and resources.
5. Comprehensive Vulnerability Coverage: The AI system demonstrates an ability to identify a broad spectrum of critical vulnerabilities, including Remote Code Execution (RCE), various forms of File Read (like XXE and traversals), SQL Injection (blind and time-based), all types of XSS (stored, reflected, blind, post-message based), Open Redirects, Exposed Secrets, Server-Side Request Forgery (SSRF), Server-Side Template Injection (SSTI), Cache Poisoning, and IDORs (though validation for IDORs is noted as still being improved).
6. Ethical Boundaries and Scope Enforcement: The system incorporates a proxy and a policy checker to ensure strict adherence to scope and ethical guidelines. It prevents unwanted actions like post-exploitation or database record deletion, making it suitable for bug bounty programs like HackerOne where such boundaries are critical.

These findings collectively illustrate a significant advancement in automated security, moving beyond basic scanning to intelligent, validated, and ethically constrained AI-driven penetration testing.

Technical Deep Dive

▶ Watch: AI's freedom in task execution (6:00)

The autonomous pentest AI developed by Expo is a sophisticated, multi-layered system designed to replicate and enhance the capabilities of human penetration testers. At its core are several interconnected components, each playing a crucial role in the vulnerability discovery and validation lifecycle.

The central orchestrator is the Coordinator. This component acts as the "manager of pentesting," analogous to a human lead pentester. Its responsibilities include:

• Analysis Priorities: Determining which areas of an application or which vulnerability types to prioritize based on context.
• Reconnaissance and Discovery: Initiating the initial information gathering phase, identifying assets, and mapping the application's attack surface.
• Task Assignment: Assigning specific vulnerability hunting tasks to individual Agents.
• Scope Management: Utilizing information from bug bounty programs (e.g., HackerOne) to embed boundaries into the system's operational parameters, ensuring agents only target in-scope assets and do not perform unauthorized actions like post-exploitation. The system parses program elements to precisely define these boundaries. The proxy component further enforces these boundaries, allowing only in-scope interactions.
• Contextual Decision Making: The Coordinator leverages provided documentation, such as Swagger files, to enhance its understanding of the application and improve test coverage and efficacy. It also tracks ongoing operations, pushing for further exploitation if a potential vulnerability is identified but not fully realized.

Agents are the hands-on vulnerability hunters. Each agent operates within a controlled environment, having access to an "attacking machine" equipped with specialized tools. These tools are adapted to work with headless browsers and other technologies, enabling the agents to interact with web applications programmatically, simulate user actions, and inject payloads. When assigned a task, an agent explores the application, looking for patterns and weaknesses indicative of vulnerabilities. The agents are given "complete freedom" within their assigned tasks, allowing for creative exploration and adaptation, much like a human pentester. For instance, an agent tasked with finding XSS will experiment with various payloads and injection points. They are also capable of detecting complex scenarios, such as when a submitted form reflects a payload in a different part of the application, which is crucial for identifying stored XSS.

The integrity of the entire system hinges on the Validators. These components serve as a "second pair of eyes," meticulously scrutinizing findings from the agents to eliminate false positives and combat LLM hallucinations and "cheating." As Brendan's Black Hat talk elaborated, validators are critical for transforming raw AI output into reliable, actionable intelligence. For an XSS vulnerability, for example, a validator would receive the generated link with the payload. It then uses a headless browser (like Puppeteer) to visit the link and programmatically confirm if a pop-up or a specific constant message appears, thereby verifying the XSS payload's execution. This approach provides a high degree of confidence, with some validators achieving near-zero false positive rates. Each supported vulnerability type has a dedicated validator, ensuring specialized and effective verification. This is particularly challenging for complex vulnerabilities like IDORs, where validation requires intricate context management and state tracking, an area the team is actively improving.

A significant innovation highlighted is the concept of Alloy Models. Recognizing that different LLMs possess varying strengths and weaknesses, the team developed a strategy to combine them. Instead of relying on a single model, when the system needs to query an LLM, it might "flip a coin" to decide which of two distinct models to use. Crucially, the system manages and saves the entire context and history of the interaction, providing it to whichever model is chosen. The key finding here is that combinations of two different models consistently outperformed single models, with greater performance gains observed when the combined models were more diverse in their capabilities. For instance, one model might be exceptionally good at cross-site scripting (XSS) detection, while another excels at application crawling and reconnaissance. By judiciously combining them, the system achieves a more comprehensive and effective attack surface. While testing with more than two models has been explored, the primary focus and success have been with dual-model combinations. This dynamic model selection allows for a more adaptive and resilient AI, capable of tackling a broader range of challenges.

The system supports a wide array of vulnerability classes, demonstrating its versatility:

• Remote Code Execution (RCE)
• File Read vulnerabilities, including XXE (XML External Entity) and path traversals.
• SQL Injection, encompassing both blind and time-based variants.
• Various forms of Cross-Site Scripting (XSS), such as stored, reflected, blind, and post-message based XSS.
• Open Redirects.
• Exposed Secrets.
• Server-Side Request Forgery (SSRF).
• Server-Side Template Injection (SSTI).
• Cache Poisoning.
• Insecure Direct Object References (IDORs), with ongoing work to refine their validation process due to inherent complexities.

Crucially, the system is designed with strong ethical boundaries and safety controls. A policy checker actively monitors the AI's actions, blocking any attempts to perform undesirable or out-of-scope activities, such as deleting database records. The system explicitly avoids post-exploitation, focusing solely on identifying and reporting the initial vulnerability. This deliberate limitation ensures that the AI operates within the confines of typical bug bounty program rules and ethical hacking principles, preventing accidental damage or unauthorized deeper access. The human in the loop is solely for the final submission to platforms like HackerOne, due to their policy requirements, rather than for validating the AI's findings.

Finally, the team emphasized an iterative optimization process. By analyzing vast amounts of data from their runs, they identified that simply running the AI more times did not equate to better results. Instead, they focused on finding the "sweet spot" for resource allocation and run duration for different vulnerability types. For instance, finding exposed secrets might require fewer runs to scan an entire application compared to uncovering a complex RCE. This data-driven approach led to significant improvements in efficiency, allowing the AI to discover more vulnerabilities with fewer attempts and computational resources. Benchmarking is conducted using custom, non-live applications to ensure consistent evaluation and prevent issues with constantly patching targets.

Demo / Proof of Concept

▶ Watch: Handling reflected and stored vulnerabilities (6:30)

While the session was presented as a Q&A rather than a formal demonstration, the speakers provided extensive detail about the system's operational mechanics, effectively describing its proof-of-concept capabilities. They clarified that the AI itself handles the entire vulnerability discovery and validation process. For instance, they explained how the Validators utilize headless browsers like Puppeteer to confirm XSS payloads by verifying the appearance of pop-ups, serving as a concrete example of the system's automated validation in action. The speakers also detailed how Agents detect reflected payloads and how the Coordinator intelligently manages tasks and ensures scope adherence, offering a comprehensive verbal walkthrough of the AI's functionality. Although no live, interactive demo was performed, the detailed technical explanations served to illustrate how the system functions in practice and validates its findings autonomously.

Defensive Implications

▶ Watch: Current limitation: no post-exploitation (7:25)

The emergence of sophisticated autonomous pentest AIs like the one developed by Expo presents both challenges and opportunities for defenders. Organizations must recognize that these AIs can perform reconnaissance, identify attack vectors, and exploit vulnerabilities with speed and scale far beyond human capabilities.

1. Proactive Vulnerability Management: Defenders need to embrace continuous and proactive vulnerability management. Relying solely on periodic manual pentests will become increasingly insufficient. Organizations should consider integrating AI-driven scanning and testing into their CI/CD pipelines to catch vulnerabilities early, mirroring the continuous attack surface assessment performed by such AIs.
2. Robust Validation in Internal Processes: The emphasis on robust Validators by Expo highlights the importance of eliminating false positives. Defenders should ensure their internal security tools and processes also prioritize high-fidelity vulnerability reporting. This means not just identifying potential issues but also having mechanisms (automated or manual) to confirm exploitability and impact before allocating remediation resources.
3. Comprehensive Attack Surface Management: The AI's ability to perform extensive reconnaissance and identify a wide range of vulnerability types (from RCE to Cache Poisoning and Exposed Secrets) underscores the need for organizations to have a clear and constantly updated understanding of their entire attack surface, including all assets, endpoints, and data flows. Shadow IT and forgotten assets become prime targets for such autonomous systems.
4. Embrace AI for Defense: Just as AI can be used for offense, it can also be leveraged for defense. Organizations should explore using AI-powered tools for threat detection, anomaly analysis, security orchestration, and even internal automated red-teaming to discover weaknesses before malicious actors or sophisticated AIs do.
5. Strengthen Core Security Posture: The AI's broad vulnerability coverage means that fundamental security hygiene remains paramount. Ensuring proper input validation, output encoding, access controls (IDOR prevention), secure configuration, and dependency management are critical to thwarting these advanced automated attacks. The "alloy models" concept also implies that attackers might combine different AI tools for multi-faceted attacks, making a defense-in-depth strategy more important than ever.
6. Ethical AI Development and Deployment: Organizations developing or deploying AI for security purposes must adopt strict ethical guidelines and implement robust safety mechanisms, similar to Expo's policy checker and scope enforcement. This ensures that AI-driven tools operate within legal and ethical boundaries, preventing unintended harm or misuse.

In essence, the rise of autonomous pentest AI signals a shift towards a more automated and intelligent threat landscape. Defenders must adapt by adopting similar levels of automation, intelligence, and continuous scrutiny to maintain a resilient security posture.

Key Takeaways

• Autonomous AI Pentesting is Here: Expo has developed a sophisticated, multi-component AI system capable of autonomous, blackbox penetration testing, covering reconnaissance, vulnerability discovery, and validation.
• Validators are Crucial for Reliability: The Validator component is key to eliminating false positives and AI hallucinations, achieving high-fidelity results through techniques like headless browsers (e.g., Puppeteer) for objective exploit confirmation.
• "Alloy Models" Enhance AI Performance: Combining diverse LLMs (e.g., "flipping a coin" between two models) significantly outperforms single models, especially for broad vulnerability discovery, by leveraging the unique strengths of each.
• Efficiency Through Intelligent Optimization: The AI learns to optimize resource allocation and run duration based on vulnerability type, finding more bugs with fewer computational resources by identifying the "sweet spot" for different tasks.
• Broad Vulnerability Coverage with Ethical Boundaries: The system identifies a wide range of critical vulnerabilities (RCE, SQLi, XSS, SSRF, SSTI, etc.) while adhering to strict scope and ethical guidelines, preventing post-exploitation and unwanted actions.
• Human Oversight for Reporting, Not Validation: While the AI handles all aspects of discovery and validation, human intervention is currently required only for the final reporting phase to platforms like HackerOne, largely due to platform policies.

About the Speaker(s)

The insights shared during this Q&A session were primarily driven by Jo, who served as the main speaker, elaborating on the core architecture and operational aspects of Expo's autonomous pentest AI. Integral to the team's work is Brendan, who delivered a separate talk at Black Hat specifically on the critical role of validators in eliminating false positives and hallucinations from AI models. The innovative "alloy models" concept, which combines different LLMs for enhanced performance, was credited to Albert, the Head of AI for Expo, and Vincent, an AI researcher also at Expo, highlighting a collaborative effort in pushing the boundaries of AI in cybersecurity. Together, this team at Expo is at the forefront of developing intelligent, reliable, and ethically guided AI solutions for penetration testing.