AI's Bitter Lesson for SOCs: Let Machines Be Machines

AI's Bitter Lesson for SOCs: Let Machines Be Machines

Speakers: Jackie Bow, Peter Sanford

Conference: BSidesSF 2025 — April 26-27, 2025, San Francisco

YouTube: Watch on YouTube

Reading time: ~9 minutes

TL;DR

The detection and response team at Anthropic built an AI-assisted investigation platform called Clue in roughly three months using Claude as both a co-engineer and runtime investigator, without any fine-tuning or specialized ML training. Drawing on the AI research concept of the "bitter lesson" — which holds that general methods beat hand-encoded human reasoning — Jackie Bow and Peter Sanford argue that SOCs should stop trying to codify every analyst decision into SOAR playbooks and instead let foundation models reason freely over their data, with transparent tooling to verify every step.

Introduction

For fifteen to twenty years, the security operations center has operated under a familiar pressure: too many alerts, too many logs, too few analysts, and too much noise from black-box ML products that performed beautifully in vendor demos and fell over immediately in production. "I've been burned by blackbox ML where the models work in the vendor's test environments," Jackie Bow told the BSidesSF 2025 audience. "When I actually applied them in my environment, they fell over — and the vendor told me: give us samples and maybe it'll get better over time."

The opposite temptation — encoding every possible investigation path into SOAR playbooks — has its own failure mode. SOAR works well for attacks already seen before. It offers nothing for novel attack patterns, and it forecloses the creative, exploratory style of investigation that experienced analysts actually bring to the table. Bow and Sanford, presenting on behalf of Anthropic's detection platform engineering team (including colleague Jack Adamson), framed both failure modes as symptoms of the same root mistake: trying to make the machine reason the way humans reason, rather than letting the machine discover its own approach.

The Bitter Lesson and Why It Applies to Detection

▶ Watch: The bitter lesson explained (05:30)

The "bitter lesson" is a concept from AI research: attempts to encode human-specific thought patterns into models consistently underperform approaches that train models generally on large amounts of data. Early chess and Go engines that embedded human strategy lost to later engines trained purely through self-play with no encoded strategy at all — those later models invented moves humans had never tried.

The implication for security is direct. SOAR playbooks are the equivalent of encoding human strategy: they work for the known, miss the unknown, and lack any mechanism for creativity. Anthropic's Claude 3.7 release documentation put it plainly: "The model's creativity in approaching problems may exceed a human's ability to prescribe the optimal thinking process." Sanford's conclusion was simple: "Let them cook."

The practical takeaway for detection engineers is not to abandon structure entirely, but to stop over-prescribing. Give the model context, data access, and light guidance — and then let it reason. This is easier than ever now that foundation models have matured to the point where they perform well at triage tasks completely out of the box, without any organization-specific fine-tuning or reinforcement learning.

Building Clue: Architecture, Tooling, and Process

▶ Watch: Building blocks and development process (12:00)

The team built its investigation platform — named Clue, a backronym for "Claude Links Useful Evidence" — in just over three months, alongside their normal detection and response work. The stack relied on several foundational elements:

Foundation models out of the box. Rather than waiting for a purpose-built model fine-tuned on security data, they found that Claude performs well at alert investigation tasks immediately. Teams blocked on the assumption that they need extensive labeled training data can start today.

Stack compatibility. The team emphasized that the technical stack matters: tools with common languages (Python, SQL, Sigma for detection rules) and robust external APIs are essential. Legacy SIEMs that treat their internals as proprietary and expose only a UI will box teams into using whatever model the vendor packages — the blackbox problem all over again.

Claude Code as co-engineer. The team leaned heavily on Claude Code throughout development. In a demo, Bow showed Claude Code receiving a vague instruction ("add a button to show raw transcripts in the investigation UI"), autonomously browsing the codebase to understand its structure, creating an implementation plan, making code changes, and deploying the result — in one minute and thirty seconds. An experienced engineer might take thirty minutes to an hour for the same task.

MCP servers. The team highlighted Model Context Protocol (MCP) servers as especially valuable for breaking models out of their context windows and giving them structured access to external systems. They named Slack, GitHub, Semgrep, and a recently released Ghidra MCP server as particularly useful for their work.

External memory. Storing prior investigation reports in a structured file system — not necessarily a complex vector database or semantic graph — gives the model a queryable knowledge base of past investigations. Bow described this as a "meta-signal" that dramatically improves future investigations by providing organizational context.

How Clue Works in Production

▶ Watch: Clue architecture and alert investigation workflow (22:00)

The core workflow began with a simple question: if they handed an alert to Claude and asked how it would investigate it, and also asked what tools it would need — could they build a functional investigation system around those answers? The answer was yes.

The process:

1. Feed Claude an alert event, the detection rule that fired, the SQL schema of available data tables, and minimal guidance on investigation approach (deliberately not over-specified).
2. Let Claude hallucinate a list of tools it wishes it had. Then ask it to implement those tools in Python, given what APIs actually exist in the environment.
3. Plug those Python tools back into the system so Claude can call them in a live investigation.
4. Claude runs the investigation, queries the data lake, generates a triage report.

The presenters were clear that using structured hallucination this way — asking Claude to invent the tools it wants — is intentional. "We don't want Claude to hallucinate events that didn't happen," Sanford said, "but we do want Claude to be creative."

Every investigation produces two outputs: a triage report with findings and enough context for a human analyst to quickly understand the situation, and a complete investigation transcript showing every query run, every data source accessed, and every step in Claude's reasoning. That transcript is the answer to the blackbox problem. "We can have actual confidence in the process and understand what the tool is doing," Bow said.

Batch Processing, Meta-Analysis, and the UEBA White Whale

▶ Watch: Batch processing and meta-analysis (32:00)

Individual alert investigation is valuable, but the team's most ambitious work is batch processing. High-confidence alerts each get their own triage report. Lower-confidence signals — which can run at roughly 100 times the volume of alerting detections — are batched together, and a separate workflow runs meta-analysis across all of them to surface patterns that would be invisible on a per-alert basis.

This architecture points toward what Bow called the "white whale" of security: UEBA — User and Entity Behavior Analysis. The promise of UEBA has been sold to the industry for over a decade: fully understand normal behavior for users and entities, and anomalies become easy to spot. The team believes their approach can actually deliver on that promise without requiring expensive purpose-built tools. "We can do entity extraction and get a really close contextual idea of what's going on in your environment, and you don't need external tools for it," Bow said.

A live demo showed the natural language investigation interface: an analyst types "here are three contractors — can you get me the logs to show if they accessed anything they shouldn't have?" The system issued queries against multiple data tables, self-corrected a SQL error, produced a complete report, and made all queries visible for verification — in two and a half minutes.

Notable Quotes

"Let them cook. The model's creativity in approaching problems may exceed a human's ability to prescribe the optimal thinking process." — Peter Sanford (08:00)

"It's vibe coding, but just in the sense that it's enjoyable — not in the sense that we're yolo-ing it." — Jackie Bow (20:30)

"We can have actual confidence in the process and understand what the tool is doing — this is a lot of where we think we have something different than the blackbox ML we talked about at the beginning." — Jackie Bow (28:00)

Key Takeaways

• Foundation models work out of the box for alert triage today — no fine-tuning, no labeled training sets required. Teams blocked waiting for purpose-built models should start experimenting now.
• Over-prescribing investigation logic in SOAR playbooks is a bitter lesson trap. Codifying human reasoning limits the model to what humans already know; general methods consistently win.
• Transparent tooling is the antidote to blackbox ML skepticism. Every query, every data source accessed, and every reasoning step in Clue's investigation is logged and reviewable — eliminating the "we don't know why it decided that" problem.
• MCP servers and external memory are powerful force multipliers that extend what a model can do beyond its context window, and provide organizational context that dramatically improves future investigations.
• A small team can build this. Clue was prototyped in three months alongside full-time detection and response work, using Claude Code for development acceleration and off-the-shelf cloud infrastructure.