Let LLM Learn: When Your Static Analyzer Actually Gets It

Let LLM Learn: When Your Static Analyzer Actually Gets It

Speaker: Hao Zhong — Security Researcher (formerly browser/OS/cloud-native exploitation; now AI security applications)

Conference: Black Hat USA 2025 — August 6-7, 2025, Mandalay Bay, Las Vegas

YouTube: https://www.youtube.com/watch?v=FPzOgf2EGQE

Reading Time: ~7 minutes

Type: Briefing

TL;DR

Existing SAST tools like CodeQL deliberately over-restrict their rules to minimize false positives, inadvertently suppressing real vulnerabilities before any LLM ever sees them. This research proposes inverting that design: use large language models during the development phase to relax and optimize SAST rules, achieving roughly three times the recall rate while maintaining near-100% precision, then apply a path-segmentation and Chain-of-Thought reasoning framework at runtime to triage the resulting flood of findings without the prohibitive per-path LLM inference cost.

Introduction

Static application security testing (SAST) and large language models (LLMs) are increasingly paired in commercial security products, but most integrations take the same basic form: SAST scans first, LLM filters after. The assumption is that SAST produces the candidate findings and the LLM decides which ones are real. Researcher Hao Zhong challenges the premise itself: if SAST rules are tuned to suppress false positives, they are also suppressing true positives — and no amount of LLM sophistication downstream can recover vulnerabilities that were never surfaced in the first place.

This talk, co-authored by researchers from both AI and security backgrounds, maps the design space of SAST+LLM integrations, identifies the failure modes in each, and presents a new architecture that brings LLMs into the rule design phase rather than confining them to post-scan triage.

Three Architectures for Combining SAST and LLMs

▶ Watch: Three Design Patterns for SAST+LLM (00:00)

Zhong categorizes current approaches into three models:

AI-Enhanced Design places SAST as the front-end scanner and the LLM as a back-end filter. This is the most common pattern in commercial products today and the easiest to ship — but it inherits all of SAST's recall limitations. CodeQL alone has over 500 pull requests dedicated to reducing false positives, and that aggressive tuning means real vulnerabilities are structurally invisible to any LLM sitting downstream.

AI Explorer Design inverts the relationship: the LLM leads exploration and the SAST tool verifies results, e.g., checking control-flow correctness. The problem is cost. Agent-style tool-calling — grep, read, file traversal — is even slower than SAST rules for systematic code exploration. What should be a rule-based task becomes an expensive, non-deterministic agent loop.

AI-Native Design has the LLM act as the entire scanner. It faces coverage gaps from knowledge cutoffs, hallucination, and agent personality variance, and it compounds the cost problem because every reasoning path requires separate LLM invocations.

The Core Insight: Relax Rules at Development Time

▶ Watch: LLM-Driven Rule Optimization Framework (08:01)

Rather than plugging the LLM into the runtime scan pipeline, the researchers adapt the optimization framework Google published roughly two years ago for using LLMs as iterative optimizers. Mapped onto SAST lifecycle:

1. The LLM acts as a QL optimizer: it generates candidate CodeQL rule modifications (relaxed source/sink definitions, loosened domination constraints) and produces a first-pass QL file.
2. The QL file is compiled and run against a public test suite (C/C++ taint analysis rules in the prototype).
3. Compilation and execution logs are fed back to the LLM for re-optimization — a closed loop that iterates until recall and precision targets are met.

Two engineering challenges complicated early iterations. First, LLM-generated CodeQL was frequently contaminated with Python or SQL syntax, causing compilation failures and death-loop iterations. The fix was strict prompt constraints limiting the scope of generated code — particularly viable for C/C++ taint analysis where source/sink definitions are less complex than in framework-heavy languages. Second, context isolation matters: feeding raw source code to the evaluator model degraded performance. Instead, the model is given structured references to the test cases that failed (file paths, line numbers) without raw code, preserving signal while controlling context size.

The result: CodeQL's precision held near 100% (confirming the team's respect for CodeQL's original tuning work), while recall improved approximately three times over the baseline rule set. The optimized rules and the agent implementation are being open-sourced.

The Runtime Problem: Triage at Scale

Relaxed rules produce more true positives — and proportionally more redundant findings. A single taint analysis over a large C/C++ codebase can generate thousands of paths that differ only in minor branch variations. Processing each path with a separate LLM call is economically and temporally infeasible.

▶ Watch: Path Segmentation and Human-Like Reasoning (14:01)

The team's insight is to model how experienced human auditors actually work. Security researchers do not reason over individual data-flow traces. Instead, they identify code regions with coherent business logic, build an abstract understanding of what that region does (e.g., "this function handles a free chain for this type"), and apply that abstraction across all paths that pass through the same region.

Two causes drive redundancy in relaxed taint analysis outputs:

1. The taint engine doubles paths on small conditional branches. As branches accumulate, the number of distinct paths grows combinatorially even when the underlying logic is identical.
2. Relaxed source/sink definitions introduce ambiguous propagation points, further multiplying path counts.

Path Segmentation and the Operation Database

▶ Watch: Chain-of-Thought Item Design (26:03)

The solution is path segmentation at the outermost caller level. In C/C++, the outermost function in a taint path represents the business-logic layer — it has a complete stack frame context and is the coordination decision point. By abstracting the source chain and sink chain separately at this boundary (rather than at individual function level or full-path level), the researchers achieve two things:

• High cache hit rate: testing showed up to 80% cache hits when matching new paths to already-analyzed code regions by function call chain identity.
• Context completeness: the abstraction ignores low-level propagation details (pointer arithmetic, variable naming) while retaining enough structural context for accurate LLM reasoning.

On top of the segmented paths, the team built an operation database — a collection of Chain-of-Thought (CoT) items indexed by function call chain. Each CoT item specifies: the relevant function/instruction, variables extracted from static analysis, permitted LLM actions, and a rationale. For example, a trans_final_dereference CoT item targeting use-after-free auditing defines what fields to check, under what conditions a dereference is concerning, and what a false positive looks like. Human domain knowledge is encoded in these items, gradually growing the database as new patterns are encountered.

This transforms the system from a rule-based scanner into a chain-of-thought scanner: SAST generates highly-related code regions, CoT reasoning operates at the segment level, and the resulting operation database accumulates reusable human insight that accelerates future audits.

Notable Quotes

"SAST is naturally great at exploring the code. It's just been forced to be held back to avoid false positives. Its full power is not released."

— Hao Zhong [06:00]

"We should first try to relax the rules, not just shape to the agentic problem-solving strategies."

— Hao Zhong [06:00]

"We could increase the recall rate to about three times better — I'm still very shocked by that."

— Hao Zhong [12:01]

"Humans don't behave like machines, so we do not force larger models to solve like machines."

— Hao Zhong [28:03]

Key Takeaways

• The bottleneck is upstream. SAST rules deliberately filter out borderline findings. Any LLM placed after a restrictive SAST scan is working with an already-truncated candidate set.
• LLMs belong in rule development, not just runtime filtering. Using LLMs to relax and optimize CodeQL rules in a closed evaluation loop tripled recall while keeping precision near perfect.
• Path segmentation solves the scale problem. Grouping taint paths by outermost-caller context achieves ~80% cache hit rates, making LLM-assisted triage feasible at scale.
• An operation database encodes human expertise. CoT items aligned to specific vulnerability operations (free chains, dereferences) allow human domain knowledge to persist and compound across audits.
• Rule-based computation is faster and more reusable. For code exploration tasks that can be captured as rules, rules should be preferred over agent tool-calling, which is slower, more expensive, and less reproducible.
• Open-source release. The team is releasing both the LLM-based rule optimizer and the runtime segmentation framework, providing a practical starting point for organizations that want to apply this methodology to their own codebases.

Slides were not listed as available for this talk.