Taming LLMs to Detect Anomalies in Cloud Audit Logs

Taming LLMs to Detect Anomalies in Cloud Audit Logs

Speakers: Yigael Berger

Conference: fwd:cloudsec North America 2025

YouTube: https://www.youtube.com/watch?v=b-MF3yGk3zQ

Overview

Yigael Berger, Head of AI at Sweet Security, presented a practical method for fine-tuning GPT-2 on cloud audit log data to build an anomaly detection engine that can distinguish routine DevOps activity from potentially malicious behavior. The core insight is elegant: rather than using an LLM to generate new tokens, the technique uses the LLM "in reverse" -- feeding it normalized CloudTrail data and reading the per-token likelihood scores to identify which log entries the model finds surprising. These anomaly scores are then fed as additional context into a larger, off-the-shelf LLM prompt to dramatically reduce false positives in cloud security detection. The research, co-developed with colleague Ido Kos, represents a production-ready approach that is accessible, inexpensive, and requires no machine learning expertise to implement.

Background

▶ Watch: Why anomaly detection matters: DevOps and attackers look identical in cloud logs (2:00)

Cloud environments present a fundamental challenge for security detection: the operations performed by legitimate DevOps teams and malicious actors are identical at the API level. Creating an IAM role, modifying a security group, or assuming a role look the same in CloudTrail regardless of intent. Traditional rule-based detection engines can catch known patterns but quickly become cluttered, difficult to maintain, and prone to false positives as environments grow in complexity.

The first generation of LLM-assisted detection -- feeding raw log data into a prompt and asking GPT-4 or Claude to identify malicious activity -- works reasonably well for scoring and explanation. These models can break events into storylines, identify smoking guns, and provide severity scores. However, they lack awareness of what is normal for a specific environment. A suspicious-looking API call that has occurred daily for three years in your environment should be treated very differently from the same call appearing for the first time, and off-the-shelf LLMs have no way to make that distinction.

Key Findings

▶ Watch: Using LLMs in reverse: reading per-token anomaly scores instead of generating... (8:00)

The research demonstrates that a small, fine-tuned GPT-2 model (approximately 124 million parameters) can effectively learn the patterns of a specific cloud environment's audit logs and flag irregular entries with high precision. The key findings include:

Training requires only 100,000 log samples and takes approximately one hour on a single GPU (or several hours on a Mac CPU). No labeled data or machine learning expertise is required -- the training is entirely self-supervised.

The technique exploits the LLM's natural language modeling capability in a novel way. After training on normalized CloudTrail data (what Berger calls "Cloudish"), the model develops an understanding of which sequences of events, regions, API calls, and response codes are typical. When encountering an unusual pattern -- such as an AssumeRole from an unexpected region following a ListBuckets call -- the model produces high surprise scores (visualized as red tokens on a heatmap).

The anomaly detection separates a tiny number of irregular patterns from the vast majority of normal log data, providing a clean signal that can be fed into downstream detection systems.

When the anomaly scores are combined with a standard LLM prompt (the two-layer approach), the resulting detection engine produces significantly fewer false positives than either approach alone. The off-the-shelf LLM gains environment-specific context it could not otherwise possess, while the fine-tuned model provides the statistical baseline.

Technical Deep Dive

▶ Watch: Days of the week example: Thursday in wrong position turns red (10:00)

The technique builds on a fundamental property of transformer models: for any input sequence, the model produces a probability distribution over all possible next tokens at every position. Berger demonstrated this using an open-source Georgia Tech visualization tool for GPT-2, showing that for the input "My name is," the model's top prediction is "John" with a specific probability, but it also assigns probabilities to every other token in its vocabulary.

By comparing the actual token at each position against the model's predicted probability for that token, you generate a per-token anomaly score. Tokens that align with the model's expectations (high probability) are "yellow" (normal); tokens the model finds unlikely are "orange" (unusual) or "red" (highly anomalous). This works not just for the final token but for every token in the input sequence.

The normalization step is critical. Raw CloudTrail JSON is not a language GPT-2 was trained on. Berger extracts key signals from each log entry -- zone, event name, source, and response code -- into a simplified format. Before fine-tuning, feeding this "Cloudish" to GPT-2 produces all-red output because the model has never seen this data format. After training on 100,000+ samples from the target environment, the model learns the typical patterns and can distinguish normal sequences from anomalous ones.

The training loop is standard self-supervised language model fine-tuning: the model reads normalized log sequences and adjusts its weights to predict the next token more accurately. No labeling, feature engineering, or anomaly tagging is required.

The two-layer architecture works as follows: Layer 1 (GPT-2, fine-tuned) processes incoming log sessions and produces per-token and per-session anomaly scores. Layer 2 (off-the-shelf LLM like GPT-4 or Claude) receives the original log data plus the anomaly annotations and generates final security assessments with dramatically improved accuracy. The anomaly context tells the large model "pay attention to this event -- it's never happened before in this environment," enabling much more precise detection.

One practical consideration Berger mentioned: when a new irregular pattern appears, it needs to be suppressed until the model can be retrained to incorporate it, preventing repeated alerting on newly observed but legitimate activity.

Demo / Proof of Concept

▶ Watch: Heat map visualization of anomalous CloudTrail events after fine-tuning (14:00)

Berger used a live, browser-based interactive demonstration of the Georgia Tech GPT-2 visualization tool to show token prediction and probability distributions. He demonstrated the "days of the week" example (Monday, Tuesday, Wednesday shows all-yellow; swapping Thursday and Wednesday causes Thursday to show as red) to build intuition about the per-token anomaly metric. He then showed a CloudTrail heat map illustrating how the fine-tuned model flags specific anomalous events like unexpected AssumeRole calls and surprising AccessDenied responses.

Defensive Implications

▶ Watch: Two-layer approach: combining anomaly scores with off-the-shelf LLM prompts (16:00)

This approach offers defenders a practical path to environment-specific anomaly detection without requiring machine learning teams or expensive infrastructure. Security teams should consider fine-tuning GPT-2 (or similar small open-source models) on their own CloudTrail data to build a baseline of normal activity. The resulting anomaly scores can augment existing SIEM rules and LLM-based detection prompts, reducing false positive rates that otherwise erode analyst trust.

Organizations should plan for periodic retraining as their environments evolve, and implement suppression mechanisms for newly observed patterns to prevent alert fatigue. The one-model-per-organization approach (rather than per-account) is recommended for most deployments, as a slightly larger model can capture cross-account nuances effectively.

The broader implication is that the barrier to building custom, environment-aware anomaly detection has dropped dramatically. This is no longer a capability reserved for organizations with dedicated data science teams.

Key Takeaways

• Fine-tuning GPT-2 on 100,000 CloudTrail log samples creates an effective, environment-specific anomaly detection engine that requires no labeled data or ML expertise
• The technique uses LLMs "in reverse" -- reading per-token probability scores rather than generating new tokens -- to quantify how anomalous each log entry is
• Combining fine-tuned anomaly scores with off-the-shelf LLM prompts creates a two-layer system that dramatically reduces false positives
• Training takes about one hour on a single GPU and can run on consumer hardware, making this accessible to any security team
• One model per organization is sufficient; there is no need for per-account training
• The approach is production-ready and deployed at Sweet Security for customer environments

About the Speaker(s)

Yigael Berger is the Head of AI at Sweet Security, where his role focuses on unlocking new cybersecurity capabilities enabled by LLMs. He co-developed this approach with colleague Ido Kos, who conceived the original idea. Together they spent several months refining the technique into a production-ready system. Berger credited Kos, who was unable to attend the conference, during his presentation.