Can a General-Purpose LLM Do Tier-1 Triage?

Can a General-Purpose LLM Do Tier-1 Triage?

Speakers: Romulo Rocha, TandemTrace

Conference: FIRSTCON

YouTube: https://www.youtube.com/watch?v=w2efXyiUuHk

Overview

In an era where Security Operations Centers (SOCs) grapple with an overwhelming deluge of alerts and persistent analyst burnout, the promise of Artificial Intelligence (AI) for enhanced threat detection and response is often touted as a panacea. Romulo Rocha of TandemTrace, a seasoned blue team and SOC operations veteran, cut through the hype in his FIRSTCON talk, "Can a General-Purpose LLM Do Tier-1 Triage?". Rocha presented compelling research on building and evaluating AI agents powered by Large Language Models (LLMs) for the critical task of Tier-1 security alert triage and campaign correlation.

Rocha's motivation stemmed from a healthy skepticism towards the "magical solutions" often marketed in the security space and a personal drive to understand the practical capabilities and limitations of AI agents. His research provides a much-needed empirical look into whether general-purpose LLMs, when properly architected into agents, can effectively augment human analysts in identifying and correlating security incidents. The talk not only demonstrated the potential for LLMs to perform complex reasoning tasks like identifying malicious campaigns but also exposed crucial trade-offs in model performance, the misleading nature of conventional metrics in a SOC context, and the paramount importance of robust engineering and observability.

Background

▶ Watch: Speaker introduction and research motivation (0:00)

The modern SOC operates under immense pressure. Analysts face a constant barrage of alerts, leading to alert fatigue, high stress, and significant burnout rates. This environment creates a fertile ground for the adoption of AI-powered products designed to automate or assist with initial alert triage. However, as Rocha noted, public, empirical evaluations of these products' effectiveness often remain scarce, leaving security practitioners to navigate a landscape of marketing claims without clear, independent data.

Rocha's research was catalyzed by the maturation of reasoning models, exemplified by the release of GPT-4 (and subsequent versions like GPT-5 mini used in his experiments), and the rapid development of AI agent architectures. These advancements offered the potential for LLMs to move beyond simple question-answering towards more complex, multi-step decision-making processes, mirroring the analytical workflow of a human Tier-1 analyst. A fundamental challenge in SOC operations, which this research implicitly addresses, is the inherent class imbalance problem: the vast majority of alerts are benign noise or false positives, while true threats are rare. Compounding this is cost asymmetry, where the cost of missing a true threat (a false negative) is orders of magnitude higher than the cost of investigating a benign alert (a false positive). Rocha highlighted historical research from the IDS community (specifically by Gaffne & Villa in 2001) that recognized these issues decades ago, emphasizing that traditional accuracy metrics often fail to capture the true operational impact in security.

Key Findings

▶ Watch: Detailed agent architecture and workflow tools (5:30)

Rocha's research yielded several critical insights into the viability and challenges of deploying LLM agents for Tier-1 triage:

1. **LLMs Can Correlate Campaigns:** The most significant finding was that properly engineered LLM agents are indeed capable of identifying and correlating alerts belonging to specific attack campaigns. His synthetic data experiments showed that both GPT-4 mini and GPT-5 mini agents successfully clustered true positive alerts into campaigns, demonstrating a fundamental ability to reason across disparate security events.

1. Model Trade-offs are Crucial:

• GPT-4 mini proved to be faster and significantly cheaper (46% less costly, 47% faster per investigation) than GPT-5 mini. It initially exhibited higher "label accuracy" (correctly identifying true positives and negatives). However, GPT-4 mini's performance degraded sharply under temporal stress, meaning it suffered from a high rate of false negatives (missed threats) when asked to correlate events across wider time windows (e.g., 90 or 180 days).
• GPT-5 mini, while slower and more expensive, demonstrated superior recall (fewer false negatives) across all temporal windows. It consistently caught almost all true positive alerts, even over extended periods. The trade-off was a higher rate of false positives (over-aggregation of noisy alerts into campaigns), leading to a lower initial "label accuracy" score.

1. Traditional Metrics are Misleading in SOC: Rocha powerfully illustrated that relying solely on standard metrics like "label accuracy" can be detrimental. Because of the cost asymmetry in security (false negatives are far more damaging), a model that prioritizes recall over precision, even if it generates more false positives, might be operationally superior. His proposed cost-weighted triage metric, which assigns a significantly higher penalty to false negatives (e.g., a false negative costs 50 times more than a false positive), revealed GPT-5 mini as the "winner" in minimizing overall risk, particularly for scenarios where missing a threat is unacceptable.

1. Engineering Trumps Raw Model Intelligence: A pivotal takeaway was that the reliability and assertiveness of the AI agent were heavily influenced by deterministic engineering efforts, not solely the LLM's inherent capabilities. Improvements such as precise tool typing using PyDantic, crafting clear prompts, and defining architectural guardrails within the agent's workflow significantly enhanced accuracy and reduced hallucinations. Rocha noted these engineering improvements increased accuracy by "nine points in some rounds" during the pre-tuning phase.

1. Calibrated Confidence Matters: GPT-5 mini's ability to provide more "calibrated confidence" scores, spreading its confidence across a wider range rather than clustering everything at high confidence like GPT-4 mini, was identified as a valuable feature. This allows human analysts to set more nuanced thresholds for investigation and better understand the agent's certainty.

Technical Deep Dive

▶ Watch: Campaign detection results and initial model tradeoffs (16:00)

Rocha's research involved building a sophisticated AI agent designed to mimic the workflow of a Tier-1 SOC analyst. The agent's architecture, inspired by the concept of a "robot" with a brain, tools, and memory, was constructed using several key components:

• Agent Framework: The core of the agent was built using LangGraph, a Python framework that allows developers to define and orchestrate complex, stateful multi-actor applications with LLMs. This enabled Rocha to design a structured workflow for the agent, where each "node" in the graph represents a specific step in the triage process (e.g., "select alerts," "Q-scan," "prioritize," "create case," "mark false positive").
• Memory and Data Storage:
• ChromaDB served as a vector database, pre-indexing all alerts. This allowed the agent to perform semantic search for similar alerts or campaigns, a crucial capability for correlation.
• SQLite was used for persistence, managing the agent's context memory and sharing states between different nodes of the LangGraph workflow.
• Embedding Model: Minim was employed as the embedding model to convert alert data into numerical vectors suitable for storage and retrieval in ChromaDB.
• Observability and Debugging: Langsmith, a platform for LLM application development, was instrumental. It provided end-to-end traceability of the agent's execution, allowing Rocha to visualize the agent's decision-making process, tool calls, and conversational loops in a waterfall-like interface. This was critical for debugging, calibration, and understanding why the agent made certain decisions.
• LLMs: The experiments primarily utilized GPT-4 mini and GPT-5 mini (presumably referring to models like gpt-4o-mini and gpt-4o, or similar cost-optimized versions available at the time of research), chosen due to budget constraints.

The agent's operational loop involved receiving a query (an alert), pushing it to the LLM with a system prompt and context memory, and then the LLM deciding whether to answer directly or call a specific tool (e.g., a database API for similar alerts). This process would iterate until the LLM determined it had sufficient information to provide a final answer or action.

A crucial distinction was made between the deterministic and non-deterministic aspects of the agent. The LangGraph code itself, where limits and guardrails are set, represents the deterministic part. This is where engineering directly influences behavior. The non-deterministic part is the LLM's "brain"—its decision-making, reasoning, and confidence levels. Rocha emphasized that significant improvements in agent assertiveness and accuracy came from refining the deterministic elements, such as using PyDantic for strict tool typing and schema validation. This prevented the LLM from "hallucinating" tool inputs (e.g., trying to use a hostname as an alert ID) and guided it towards more effective use of its available functions.

Synthetic Dataset Creation:

A major hurdle for this research, and a common challenge in the security domain, was the lack of publicly available, labeled datasets of true positive security alerts. To overcome this, Rocha innovatively created a synthetic dataset by prompting another LLM to generate alerts. To ensure realism and utility, he structured this synthetic data to:

• Adhere to TheHive schema, a popular open-source incident response platform, for consistent data structure.
• Incorporate elements from Sigma rules and Atomic Red Team techniques, ensuring the generated alerts contained realistic observables and attack patterns.
• The dataset comprised 100 alerts per round: 70% were intentionally "noisy" (false positives), and 30% were "true positives," split across two distinct attack campaigns (15 alerts per campaign).
• Experiments were conducted across three temporal windows: 30 days, 90 days, and 180 days. This allowed Rocha to test the agents' ability to correlate events over varying time spans, directly addressing the question of whether LLMs could maintain performance under "temporal stress."

Demo / Proof of Concept

▶ Watch: Introducing cost-weighted triage for SOC metrics (21:00)

While Romulo Rocha did not perform a live demonstration of the agent's code execution during his talk, he effectively conveyed the agent's capabilities and output through several visual aids. He presented trace visualizations from Langsmith, showcasing the multi-step reasoning process, tool calls, and conversational loops the agent engaged in during an investigation. Furthermore, he displayed example reports generated by the agent, which included a clear statement of the identified campaign (e.g., a ransomware campaign), a confidence score (e.g., 95%), and a detailed rationale for its decision. Rocha also mentioned that the full code for his research would be released on GitHub, allowing others to replicate, experiment with, and learn from his agent architecture.

Defensive Implications

▶ Watch: Q&A: Database paradigms for agent tooling (31:40)

Rocha's research offers several crucial implications for security defenders considering the integration of LLM agents into their SOC operations:

1. Augmented Triage, Not Autonomous: The most critical takeaway is that LLM agents are currently best suited for augmented triage, not fully autonomous decision-making. They can significantly assist Tier-1 analysts by correlating alerts and providing initial investigations, but human oversight and validation remain indispensable. The goal is to enhance analyst efficiency, not replace human judgment.

1. Increased Pipeline Complexity: Adopting LLM agents will inevitably increase the complexity of the SOC pipeline. Organizations must consider:

• Review and Validation: Who reviews the agent's output? How are its decisions validated?
• Improvement Loop: How are agents improved? This requires dedicated resources for tuning prompts, adjusting tools, and refining the underlying architecture.
• AI Engineer Requirement: It is essential to have an AI engineer or a team member with a deep understanding of AI agent architecture and LLMs. This expertise is needed whether building agents internally or evaluating third-party products.

1. Skepticism and Validation are Paramount: LLMs are "beautiful machines that create beautiful stories." They can generate convincing rationales, even for incorrect aggregations. SOC analysts must be trained to doubt everything that comes from an LLM agent or product. This necessitates pushing vendors for clear confidence level scores and providing analysts with the tools and training to question and deep-dive into the agent's reasoning.

1. Non-Negotiable Observability: Robust observability is paramount for any LLM agent in production:

• End-to-End Tracing: Tools like Langsmith are vital for understanding the agent's multi-step decision process, tool calls, and internal dialogues for audit and debugging purposes.
• Metrics: Continuous monitoring of operational metrics, such as investigation speed, cost per investigation, and the rate of false positives/negatives, is essential.
• Versioning: Versioning of prompts, schemas, and agent architectures is critical, as even minor changes can drastically alter performance.
• Audit of Decision Quality: Regular audits are needed to detect hallucinations, biases, or unexpected behaviors.
• Drift Monitoring: LLM models from providers like OpenAI can change over time. Organizations must continuously monitor for model drift and assess its impact on agent behavior and output quality.

1. Focus on "Easy Wins" First: For organizations new to LLM agents, Rocha advised starting with lower-risk applications, such as:

• Document Generation: Automating reports or internal documentation.
• Chatbots: Internal helpdesks or knowledge retrieval.
• Summarization: Summarizing shift handovers or incident timelines.

He cautioned extreme vigilance and supervision for high-stakes tasks like escalation or fully autonomous Tier-1 triage.

Key Takeaways

• **LLMs can perform Tier-1 triage and correlate security campaigns**, but their effectiveness is subject to significant trade-offs between different models.
• Traditional accuracy metrics are insufficient for SOC operations. A cost-weighted triage approach, prioritizing the minimization of costly false negatives, is essential for truly evaluating agent performance and operational risk.
• Agent reliability stems more from robust engineering than raw LLM intelligence. Meticulous design of deterministic guardrails, tool typing, and prompt engineering significantly enhances agent accuracy and reduces undesirable behaviors.
• Augmented triage, not autonomous, is the current reality. LLM agents should serve as powerful aids to human analysts, requiring continuous human oversight, validation, and skepticism.
• Comprehensive observability, versioning, and continuous monitoring are non-negotiable for deploying and maintaining LLM agents in a production SOC environment.
• An AI engineer is a critical asset for any organization looking to leverage LLM agents, whether building in-house or evaluating vendor solutions.

About the Speaker(s)

Romulo Rocha, from Rio de Janeiro, Brazil, is a seasoned security professional with around 18 years of experience, predominantly within blue teams and SOC operations. He describes himself as a "sock guy" – curious, but not a mathematician, machine learning specialist, or scientist. Rocha approaches new technologies with a healthy skepticism towards "magical solutions," which directly motivated his research into AI agents. He is a member of the FIRST community and other special interest groups, and this was his second talk at a FIRST conference. His practical, hands-on approach to understanding complex security challenges underscores the value of his empirical research.