Sometimes Simpler is Better: A Comprehensive Analysis of State-of-the-Art Provenance-Based Intrusion Detection Systems
Tristan Bilot
34th USENIX Security Symposium (USENIX Security '25) · Day 3 · Network Security 4: Internet and Beyond
In the ever-evolving landscape of cybersecurity, detecting sophisticated attacks requires robust and intelligent systems. Provenance-based Intrusion Detection Systems (PIDS) have emerged as a promising approach, leveraging system-level causality graphs to trace malicious activities. However, the complexity of state-of-the-art PIDS, often relying on Graph Neural Networks (GNNs) and sophisticated anomaly detection techniques, raises questions about their practical deployability and actual effectiveness. This talk, presented by Tristan Bilot, a PhD student at University Pra and visiting student at UBC, delivers a comprehensive analysis of eight recent, top-tier PIDS, revealing critical shortcomings in their design, evaluation, and practical utility.
AI review
Rigorous systems security research that does the unsexy but necessary work: re-implementing eight GNN-based PIDS in a unified framework, exposing evaluation methodology rot across the subfield, and dropping a simple baseline (VLOGS) that embarrasses the competition across nine DARPA datasets after 400+ GPU-compute-days of validation. The metric critique alone — threshold dependency, per-attack vs. aggregate TP counting, class imbalance masking — is worth the runtime.