Universal and Context-Independent Triggers for Precise Control of LLM Outputs

Universal and Context-Independent Triggers for Precise Control of LLM Outputs

Speakers: Jiasuo Liang and Guancheng Li (Security Researchers, Tencent Xuanwu Lab)

Conference: Black Hat USA 2025 — August 6-7, 2025, Mandalay Bay, Las Vegas

YouTube: https://www.youtube.com/watch?v=W8zzpTGVHRE

Reading time: 7 minutes

Type: Briefing

TL;DR

Researchers from Tencent Xuanwu Lab developed "universal adversarial triggers" — short, model-specific token sequences that, when injected into any prompt, force an LLM to output exactly what an attacker specifies, regardless of the surrounding context. The technique achieves roughly 70% success across diverse prompts and payloads, works against open-source models including Qwen, Llama, and DeepSeek Small, and was demonstrated achieving remote code execution against AI coding agents — no knowledge of the application's system prompt required.

Introduction

Prompt injection has evolved from a curiosity to a critical attack vector as LLMs have moved from standalone chatbots to autonomous agents with access to email, code execution, browsers, and external tools. Early attacks relied on persuasive text — telling the model to "ignore previous instructions" or roleplay as an unrestricted persona. Those techniques are manual, fragile, and context-dependent. They require the attacker to understand the application's system prompt and craft a new attack for every scenario.

Jiasuo Liang and Guancheng Li of Tencent Xuanwu Lab presented a fundamentally different approach at Black Hat USA 2025. Their universal adversarial triggers are not persuasive text — they are optimized token sequences that directly manipulate the mathematical internals of transformer models to force a precise output. A single trigger, trained once on a given open-source model, works across arbitrary system prompts, arbitrary injection locations, and arbitrary payloads — including the kind of precisely-formatted shell commands and XML structures that AI agents need to execute actions.

The implications for AI agent security are significant: an attacker who embeds a trigger in a poisoned email, a malicious MCP server description, or any content an agent might process can achieve reliable remote code execution without ever knowing what the application is doing.

▶ Watch: Introduction and Prompt Injection Background (00:00)

The Limitations of Traditional Prompt Injection

Traditional prompt injection involves two steps. First, the attacker attempts to escape the current prompt context — often by asking the model to describe its instructions or to act under a different persona. Second, they redirect the model's attention to the attacker's task. Both steps are highly context-dependent, requiring the attacker to know enough about the application to craft plausible-looking injection text.

This approach breaks down for high-precision payloads. If the goal is RCE, the attacker needs the model to output an exact shell command in a format the agent can parse and execute. A handcrafted jailbreak that produces approximate output is not enough — every character matters. The attacker also cannot reuse the same injection across different applications with different system prompts.

Liang enumerated the four properties that an ideal prompt injection would possess: universal effectiveness (the same trigger works across applications), high accessibility (even inexperienced attackers can deploy it), precise output control (the model produces exactly what the attacker specifies), and security consequences (such as system compromise). "This seemed like an impossible combination until now," he said.

▶ Watch: Traditional Prompt Injection Limitations (04:01)

How Universal Adversarial Triggers Work

The trigger consists of two short token sequences — a prefix and a suffix — that wrap the attacker's payload. The payload is the exact text the attacker wants the model to output: a shell command, a JSON structure, an XML fragment, or any other string. The trigger sequences are largely unreadable to humans, but when processed by the target model they suppress the influence of all surrounding context and cause the model to output the payload verbatim.

Conceptually, the technique is inspired by adversarial examples in computer vision — the equivalent of pixel perturbations that cause image classifiers to misidentify objects. Applied to language models, the goal is to maximize the probability that the model outputs a specified sequence of tokens, regardless of what the rest of the prompt says.

The researchers formalize this as an optimization problem: find trigger tokens that maximize the joint probability of the model outputting each payload token in sequence. Because token IDs are discrete values, standard gradient descent cannot be applied directly. The researchers used the GCG (Greedy Coordinate Gradient) algorithm combined with the HotFlip approximation. HotFlip uses a first-order Taylor expansion in token embedding space to estimate how the loss function changes when one token is replaced with another. GCG then iterates over token positions, samples candidate replacements, tests actual loss values, and keeps the best substitution at each step. After roughly 100,000 LLM invocations across a diverse training dataset, the trigger converges.

The training data consists of public instruction-following datasets augmented with domain-specific agent conversation patterns (web and coding dialogues). A pipeline randomly selects injection locations and generates varied malicious payloads — incorrect answers, off-topic responses, and malicious commands wrapped in plain text, JSON, and XML. This diversity is what makes the resulting trigger universal: it has been optimized to overpower arbitrary context.

▶ Watch: Technical Deep Dive — GCG Algorithm (14:02)

Demo 1: Remote Code Execution via Open Interpreter

The first demonstration targeted Open Interpreter, an agent that lets users operate their computer through natural language commands. The attack scenario: an attacker sends the user a malicious email containing a shell command wrapped in the trigger prefix and suffix. The user asks Open Interpreter to check their mailbox. The agent writes Python code to read the email, retrieves the email content — and when the model processes the trigger-wrapped payload, it outputs the attacker's shell command in the exact format Open Interpreter expects for execution. The command runs, and the attacker receives a reverse shell.

From the user's perspective, the agent appeared to summarize the email. There was no prompt asking for permission to run code — the trigger caused the model to bypass any such safeguard and output directly executable content.

▶ Watch: Open Interpreter RCE Demo (08:01)

Demo 2: RCE via Malicious MCP Server in a Coding Agent

The second demonstration targeted Cline, a popular VS Code extension for AI-assisted coding. The attack vector was a malicious MCP (Model Context Protocol) server — the mechanism coding agents use to extend their capabilities with external tools.

The attack flow: an attacker publishes a seemingly benign MCP server and waits for it to accumulate users. They then update the tool's description to embed a trigger prefix and suffix around an XML payload. The XML payload includes a RequiresApproval: false flag and a shell command. Cline supports an "Execute safe command" auto-approval feature for commands the model determines are low-risk. Because the trigger forces the model to output the attacker's XML structure verbatim — including the false approval flag — the command executes without any user confirmation prompt.

Critically, sandboxing the MCP server does not help. The tool description is loaded into the prompt and processed by the model running on the user's machine; the resulting command is executed in the user's VS Code terminal, not in the sandbox. In the demo, a calculator application launched silently when the user asked Cline to describe the current project.

"The model we use is DeepSeek Small. It's an open source model," Liang noted, indicating the attack targets deployments of fine-tuned open-source models as back-ends for coding agents.

▶ Watch: Cline MCP Server RCE Demo (10:02)

Performance, Transferability, and Limitations

Across testing on Qwen 2.5-7B-Instruct, Llama, and DeepSeek Small, the trained triggers achieved approximately 70% attack success rate across diverse prompt contexts and payload types. Transferability within model families was meaningful — a trigger trained on Llama 3.1-8B maintained around 60% success on Llama 70B, and a trigger trained on Qwen 2-7B retained similar success on Qwen 2.5-7B. Transferability across model families (e.g., from Qwen to Llama) did not hold in their experiments.

The technique has notable limitations. It requires white-box access to model weights and gradient information to train triggers, meaning it cannot be directly applied to proprietary closed-source APIs like GPT-4 or Claude. The resulting trigger strings are incoherent to humans and can be detected by perplexity-based filters. Training requires substantial compute — approximately 100,000 LLM invocations. And triggers do not transfer across model families, limiting the blast radius of any single trigger.

An audience member asked whether cross-family generalization might eventually be possible by training on multiple models simultaneously. Liang acknowledged it as an open research question: "That will maybe work on private models."

▶ Watch: Results, Limitations, and Transferability (18:03)

Notable Quotes

"I will show you how a single magic string can reliably force language models to output exactly what the attacker wants." — Jiasuo Liang [00:00]

"This seemed like an impossible combination until now." — Jiasuo Liang, on the four ideal properties of a prompt injection [04:01]

"Prompt injection no longer just causes misinformation, but enables code backdoors, data theft, and even complete system compromise." — Jiasuo Liang [02:01]

"LLMs are not trustworthy by default. We should always run agents in a sandbox and implement proper security controls." — Jiasuo Liang [22:04]

Key Takeaways

• Universal adversarial triggers decouple the attack from the application: A single trained trigger works across diverse system prompts and injection locations, eliminating the need to craft bespoke injections per target.
• Precisely-formatted RCE payloads are now achievable: The triggers force models to output exact XML or shell commands, enabling code execution through AI agents like Open Interpreter and Cline without user confirmation.
• Malicious MCP servers are a high-risk supply chain vector: Poisoned tool descriptions processed by an agent's model can execute attacker commands even when MCP servers are sandboxed.
• 70% success rate across diverse contexts: Against tested open-source models, triggers achieve reliable output control — lowering the skill floor for prompt injection significantly.
• Defenses must go beyond sandboxing: Perplexity-based input filters can detect the incoherent trigger strings, but architectural controls — agent isolation, strict output parsing, and minimal permission scopes — are necessary layers of defense.

Slides

No slides PDF was available for this briefing.