Chimera: Harnessing Multi-Agent LLMs for Automatic Insider Threat Simulation

Chimera: Harnessing Multi-Agent LLMs for Automatic Insider Threat Simulation

Speakers: Jiongchi Yu

Conference: NDSS Symposium 2026

YouTube: https://www.youtube.com/watch?v=i6TPk47brhY

Overview

This talk introduces Chimera, the first multi-agent LLM framework for automated insider threat simulation. The system addresses a critical gap in insider threat detection: the lack of high-quality, realistic, large-scale, and adaptive datasets for training and evaluating detection models. Chimera deploys LLM-powered agents as employees within a simulated organization, generating both benign work activities and adversarial insider behaviors with full semantic context.

The resulting dataset, Chimera Log, comprises a 20-person organization simulation over one month, generating 20 billion benign log entries and 5 billion attack entries across six log modalities: application logs, HTTP/browser histories, email communications, file access, system logs, and network traffic. Human expert evaluation using five-point Likert scales confirmed that Chimera Log is significantly more realistic than existing synthetic datasets. The framework supports 12 individual attack types and 3 hybrid attacks structured around MITRE ATT&CK TTPs extracted from real FBI and US Attorney Office prosecution records.

Background

▶ Watch: Insider threat landscape: $70M average cost per organization (0:00)

Insider threats represent one of the most challenging detection problems in enterprise security. Unlike external attackers, insiders operate within the trusted boundary, making their malicious activities difficult to distinguish from legitimate work. According to recent reports, the average cost of insider threats per organization is $70 million. Insider threats fall into three categories: malicious insiders (intentional attackers), compromised insiders (hijacked legitimate accounts), and negligent insiders (unintentional security violations).

Detection methods have evolved from traditional rule-based approaches to machine learning-based systems, but all share a fundamental dependency on high-quality training data. This data problem manifests in several ways:

Real organizational data is privacy-sensitive and cannot be released externally. Labeling is extremely costly. Synthetic datasets like the widely-used CERT dataset from CMU model activity patterns but lack semantic information -- emails are just combinations of random words without realistic content. Real-world experimental datasets using human participants produce realistic data but are inherently limited in scale.

The core paradox is that organizations need large volumes of realistic, up-to-date data to address distribution shift and concept drift in their detection models, but existing data sources cannot provide this. Chimera uses multi-agent LLM systems as a natural solution: each agent represents an employee, generating contextually rich activities that produce realistic log data at scale.

Key Findings

▶ Watch: Multi-agent LLMs as employee simulators (4:00)

Superior realism over existing datasets: Human expert evaluation by five independent security experts rated Chimera Log significantly more realistic than existing datasets across four realism dimensions. The CERT dataset, by comparison, contains no semantic information -- just word combinations without meaningful content.

Six log modalities: Chimera generates application logs (login activities), HTTP logs (browser histories), email communications, file access logs, system logs, and network traffic -- two more modalities (system logs and network traffic) than the standard CERT dataset format.

Distribution shift is a real problem: Cross-domain evaluation (training on one organizational type, testing on another) showed significant performance decreases in existing detection models. Chimera Log demonstrated improved cross-domain generalization but still suffered from distribution shift, reinforcing the need for organization-specific, frequently updated training data.

Existing detectors struggle with Chimera data: Standard insider threat detection models performed worse on Chimera data than on existing synthetic datasets, suggesting that Chimera's higher realism exposes limitations in current detection approaches.

12 structured attack types: Attack scenarios are derived from real prosecution records (FBI, US Attorney Office, data broker databases) and manually mapped to MITRE ATT&CK TTPs. Three hybrid attacks combine multiple patterns for complex multi-stage scenarios.

Technical Deep Dive

▶ Watch: Hierarchical scheduling and attack activity planning (8:00)

Chimera's architecture consists of several interconnected components:

Organizational Profiling: The first step configures the simulated organization -- structure, roles, hierarchy, access policies. This can be manually specified for a target organization or generated by the LLM itself. An example showed a generated gaming company with hierarchical structure. This profiling is highly configurable, enabling simulation of different organizational types (technology companies, financial organizations, medical institutions).

Agent Society Construction: Each agent is a context-aware entity with personality traits, role-specific knowledge, and access control settings -- not a simple tool executor. Agents maintain persistent state including their work context, social relationships, and ongoing projects. This contextual awareness enables realistic behavioral patterns.

Hierarchical Scheduling: Activity scheduling operates at multiple levels:

• Weekly scheduling: Group discussions among agents to plan meetings and set weekly goals
• Daily task scheduling: Individual agents plan their daily activities
• Progress reflection: Agents evaluate their work output and adjust future plans
• Communication-triggered updates: When agents interact, both update their schedules based on the conversation, creating organic activity patterns

Adversarial Agent Behavior: Adversarial agents function as normal employees during non-attack periods, blending into the benign activity baseline. Attack planning is integrated into the schedule update process -- the agent plans attack activities alongside normal work, making the transition between benign and malicious behavior realistic.

Simulation Execution: Simulation runs through agents executing their earliest scheduled tasks until all daily schedules complete. Time compression is configurable -- the evaluation used 20 seconds per 1 real-world second, but this can be adjusted based on budget and fidelity requirements.

Attack Structuring: 12 individual attack types and 3 hybrid attacks are structured using MITRE ATT&CK TTPs extracted from real-world prosecution records. An example showed an intellectual property theft attack with specific TTPs mapped to agent actions. Hybrid attacks combine multiple patterns (e.g., data exfiltration combined with credential compromise).

Log Collection: As agents execute activities (browsing, writing code, sending emails, accessing files), the system captures real activity logs across all six modalities, producing data that maps to standard enterprise log formats.

Demo / Proof of Concept

▶ Watch: 12 attack types from FBI and prosecutor records mapped to MITRE ATT&CK (10:00)

No live demo was performed. The evaluation deployed Chimera across three highly data-sensitive industrial scenarios: a technology company, a financial organization, and a medical institution. The resulting Chimera Log dataset consists of a 20-person organization simulated over one month, producing 20 billion benign entries and 5 billion attack entries across six log modalities.

Quantitative evaluations included temporal activity distribution analysis (comparing daily activity histograms between Chimera, CERT, and real-world data), behavior entropy measurements (confirming diversity of simulated activities), and sequence complexity analysis (confirming realistic behavioral patterns). Standard insider threat detection models were benchmarked on Chimera data, showing increased difficulty compared to existing synthetic datasets, which validates the dataset's realism.

Defensive Implications

▶ Watch: Future directions: self-evolving adversaries and human-in-the-loop (14:00)

Chimera has significant implications for enterprise insider threat programs:

On-demand training data: Organizations can deploy Chimera to generate organization-specific training data for their insider threat detection models, eliminating the dependency on outdated public datasets. The organizational profiling capability means the generated data can closely match the target environment.

Detection model validation: The finding that existing detection models struggle with Chimera data suggests current models may be overfitting to the simplistic patterns in existing synthetic datasets. Organizations should evaluate their detection models against Chimera-quality data to identify gaps.

Red team automation: Chimera can serve as an automated insider threat red team, generating realistic attack scenarios that test detection capabilities without requiring human participants. The MITRE ATT&CK TTP structuring ensures attack scenarios are grounded in real-world threat intelligence.

Addressing distribution shift: The cross-domain evaluation results demonstrate that distribution shift is a significant problem for insider threat detection. Chimera provides a mechanism to generate fresh training data as organizational structures, tools, and work patterns evolve.

Limitations: The framework currently requires manual setup for attack TTP extraction from real cases. Fully automated adversarial agent behavior and self-evolving attacks competing against defense systems are identified as future work. Real-world deployment requires significant configuration effort to achieve representative organizational profiling.

Key Takeaways

• Chimera is the first multi-agent LLM framework for automated insider threat simulation, generating realistic organizational activity data at scale
• Chimera Log dataset: 20-person organization, one month, 20 billion benign entries, 5 billion attack entries, six log modalities
• Human expert evaluation confirms significantly higher realism than existing synthetic datasets like CERT
• 12 individual attack types and 3 hybrid attacks structured around MITRE ATT&CK TTPs extracted from real prosecution records
• Existing insider threat detection models perform worse on Chimera data, suggesting overfitting to simplistic synthetic datasets
• Distribution shift across organizational domains causes significant detection model degradation, reinforcing the need for organization-specific, frequently updated training data
• Code and data to be released publicly

About the Speaker(s)

Jiongchi Yu (Jun) is a researcher at Singapore Management University working on insider threat detection and multi-agent LLM systems. Yu demonstrated strong knowledge of both the insider threat detection landscape (including the data quality challenges that have hampered the field for years) and the practical engineering of multi-agent simulation systems. The work represents a bridge between LLM capabilities and a long-standing enterprise security data problem.