AI Agents for Offsec with Zero False Positives
Black Hat USA 2025 · Day 1 · Briefings
AI agents used naively for offensive security produce an overwhelming number of false positives — a problem that compounds catastrophically at scale due to the base rate fallacy. Brendan Dolan-Gavitt presents a practical framework using deterministic validators (canaries and evidence-based checks) to drive AI-assisted vulnerability discovery toward zero false positives, demonstrating real bugs found in Apache HugeGraph, Redmine, and other widely deployed open source projects. ---
AI review
Dolan-Gavitt walked into Black Hat and dropped what is probably the most operationally mature AI-assisted bug hunting research I've seen. Not hype — theorems, scale, and 650+ confirmed vulns in the backlog. This is what responsible AI security research looks like.