Operation Pale Fire: How We Red-Teamed Our Own AI Agent

Operation Pale Fire: How We Red-Teamed Our Own AI Agent

Speakers: Josiah Peedikayil & Wes Ring, Offensive Security Team, Block

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 4, 2026, San Francisco

Watch: YouTube — Full Talk

Reading time: ~9 min

TL;DR

Block's offensive security team ran a full end-to-end red team operation against Goose, their own open-source AI agent — and achieved code execution on employee laptops via invisible Unicode characters hidden in Google Calendar invites and weaponized Goose "recipes." The operation revealed that operationalizing prompt injection for a stealthy, realistic attack is significantly harder than it looks, and the lessons are as valuable as the successful exploits.

Introduction

Operation Pale Fire is one of the most detailed accounts of what an end-to-end AI red team operation actually looks like from the inside. Josiah Peedikayil, presenting on behalf of the Block offensive security team (with collaborators Wes Ring, Hassan S., Michael Rand, and Justin Engler), walked through the full arc of the operation: the reconnaissance, the two attack campaigns, what went wrong, the eventual compromise, and the mitigations that followed.

Block's red team operates by a specific mandate: simulate external adversaries, run end-to-end operations from outside the company boundary to the objective, and place heavy emphasis on evading detection. "Our blue team is very good," Peedikayil noted. "If they find us, they will evict us from the environment." That constraint shaped every decision in the operation.

The target was Goose — an AI agent that started as an internal Block project in fall 2024 and has since been open-sourced. Goose is used by both engineers and non-technical employees, ships by default with a developer shell tool call (meaning it can run shell commands), integrates with Claude 3.5 Sonnet, Claude 3.7, GPT-4, and other models, and connects to a large internal MCP marketplace covering tools from Jira to Google Drive.

The shell tool call was the obvious angle. The MCP marketplace was the attack surface.

▶ Watch: Introduction and Goose overview (00:00)

Why Red-Team an AI Agent?

The team's stated goals clarify the scope and ambition of the operation:

1. Understand what an end-to-end AI red team operation actually looks like. Prior research on prompt injection is strong; end-to-end operational research is sparse.
2. Identify whether Goose could be exploited as an initial access vector before it was open-sourced and its source code became available to real attackers.
3. Determine whether Goose had features dangerous enough to create initial access opportunities for an external attacker.
4. Operationalize a novel prompt injection. The key word is "operationalize." Building a prompt injection in a lab is one thing. Making it stealthy, reliable, and evadable in a real enterprise environment is something entirely different.

The final objective: gain code execution on a Block employee's laptop.

▶ Watch: Goals and red team methodology (00:00)

MCP Reconnaissance: Finding the Injection Surface

The team began by surveying the internal MCP marketplace looking for a specific type of target: MCPs that ingested untrusted external text and routed it into the model's context. That is the precondition for any prompt injection attack via external content.

Google Calendar stood out immediately. The internal Google Calendar MCP at Block, as configured at the time, allowed external parties to schedule events on Block employee calendars. This is a feature of the Google Calendar API: external parties can add events directly to a target calendar. And critically, the Calendar API allows senders to disable the email notification that normally accompanies an invite. With the notification disabled, the invite silently appears on the employee's calendar with no associated email — no artifact, no alert, fewer threat hunting signals.

Additional stealth factors: the attacker's account can be omitted from the guest list entirely, so only the target employee appears on the invite. A shared calendar can be named something that looks internal — "Block Calendar" — to blend into the environment. The only real indicator of external origin is a small "Created by" field at the bottom of the invite in small italicized text. As Peedikayil noted: "Not something most people scrutinize, including myself."

The attack chain in concept: a Block employee asks Goose "What's on my calendar today?" → Goose reads the calendar via MCP → ingests the invite description (including hidden payload) → the hidden payload triggers Goose's developer shell tool call → a curl-bash command executes the attacker's payload.

▶ Watch: MCP recon and the Google Calendar finding (02:00)

Campaign 1: The Invisible Prompt Injection

The first campaign's challenge was making the prompt injection stealthy. The raw payload — long, detailed instructions telling Goose to run a curl-bash command to "update" the Calendar MCP — was visually alarming. A blue team analyst who saw a calendar invite with a wall of suspicious instructions would immediately flag it.

The solution was invisible Unicode characters. Zero-width Unicode characters exist that render as nothing to the human eye but are processed as regular characters by an LLM. Using these characters, the team converted a large, visible prompt injection into what appeared to be a completely empty calendar invite description — visually indistinguishable from a normal invite.

The technique, sometimes called ASCII smuggling, worked as intended: the invite looked clean to a human reviewer, but Goose ingested and processed the hidden instructions.

The attack flow:

1. Build malicious payload (curl-bash command to execute the attacker's infrastructure)
2. Encode the payload using zero-width Unicode characters to make it invisible
3. Send calendar invites from an external attacker account to target employees
4. Wait for a target employee to ask Goose to check their calendar
5. Goose reads the calendar, ingests the hidden instructions, executes the developer shell tool call

The invites went out. No shells arrived.

The post-mortem revealed multiple failure points. The Google Calendar MCP internally had received a major rewrite during the operation — the team's prompt injection had been tuned for the previous version and was now incompatible with how the updated MCP structured context. Model resistance also varied significantly: the injection worked reasonably well against Claude 3.5 Sonnet but poorly against Claude 3.7 and GPT models. The team had tuned for one model, so anyone using a different one was unaffected.

Most importantly, the testing environment had misled them. The team tested on their own calendars, which were sparse. A real employee calendar with 20+ meetings fills the model's context window quickly, and the single invite containing the injection got lost — the model would simply forget it before getting around to executing anything.

"Context is king," Peedikayil concluded. "Always test with a realistic context window."

▶ Watch: Campaign 1 — invisible text technique (06:00)

Campaign 2: Weaponizing Recipes via the System Prompt

The failure of campaign 1 prompted a strategic insight: a user-prompt injection is fundamentally weaker than a system-prompt injection. The team shifted to targeting Goose's recipe system.

A recipe is a Goose-specific feature — a reusable workflow packaged as a shareable URL. The URL uses a custom goose:// URL handler and encodes a JSON payload in Base64 containing version, title, description, and instructions. When a user opens the URL, Goose launches a new conversation and runs the embedded instructions. Critically: recipes go into the system prompt, not the user message. That makes a recipe-based injection fundamentally stronger than anything at the user context level.

The team verified this quickly: embedding a simple instruction to "open Chrome to a URL" in the recipe instructions field and sharing the link worked cleanly.

Delivery was the next problem. If anyone decoded the Base64, the malicious instructions would be visible. The solution: apply the same zero-width Unicode technique to the instructions field inside the recipe. Once decoded, the recipe appears to have an empty instructions field. The only tell is a cluster of zero-width characters visible in a hex dump — not something a typical developer reviewing a recipe URL would notice.

The revised attack used a modified calendar invite labeled "Rescheduled" (to evade any pattern matching on the previous campaign's format), with a hyperlink to the malicious recipe and social engineering copy inviting employees to try a new Goose workflow. The team even included a Google Meet link — initially for authenticity. That decision worked too well: on the first campaign run, employees actually joined the Meet during their blocked-off time. The second run, the team had a slide deck ready to walk through the "recipe" and encourage adoption.

Someone clicked the link. Execution was triggered. Then a typo in the payload failed it — the LLM had been autocorrecting the typo during testing, but in production there was no autocorrection. The team found the error during post-mortem.

Google Calendar's rate limiting (50 invites per day) pushed the team to add a spear-phishing track. They reached out to the Goose development team via a public channel, posing as external security researchers who had found a bug related to RTL text rendering in a recipe. They asked the dev team to reproduce it — which meant running the recipe. The dev team ran it. The curl-bash payload executed. The blue team caught them shortly after and deconflicted.

▶ Watch: Campaign 2 — recipes and social engineering (12:01)

Mitigations and Detections

The operation drove a set of concrete security improvements to Goose:

Input sanitization: Strip non-standard Unicode characters from all inputs and from recipes before they reach the model. This eliminates the zero-width character smuggling technique.

Recipe transparency: Display the complete decoded recipe content to the user before execution, with an alert if hidden or suspicious text is detected. Users now know what they're clicking before Goose runs anything.

Prompt injection detection: The team explored multiple approaches — Meta's Llama-Prompt-Guard-286M, Protect AI's deberta-prompt-injection-v2, and an internally built bad bash command detector called Project Bashcat (a BERT and n-gram classifier). LLM-based semantic evaluation of potential injections was also tested, with mixed results.

Command allowlisting: Restricting which shell commands Goose can execute reduces the blast radius of any injection that does succeed.

Google Calendar policy: Changes to how external calendar invites are handled — requiring some user acknowledgment before external invites appear on primary calendars.

▶ Watch: Mitigations (18:01)

Notable Quotes

"I want to emphasize the word 'operationalize' — yes, it's possible to make a prompt injection. But can you make one without getting caught? Can you make one that hides in plain sight? That's what we were really interested in answering." — ▶ 04:00

"Context is king. Always test with a realistic context window." — ▶ 10:00

"I don't really blame any of our users. This is a well-crafted attack." — ▶ 14:01

"Props to the blue team — they caught us shortly after execution and deconflicted with us through our red team inquiry process." — ▶ 16:01

Key Takeaways

• System prompt injection beats user context injection. Goose recipes, which load into the system prompt, were a significantly more powerful vector than calendar invite descriptions loaded into user context. When evaluating AI agent attack surfaces, the distinction between where content lands in the context matters enormously.
• Zero-width Unicode characters are a practical evasion technique. Invisible to human reviewers, processed normally by LLMs — stripping non-standard Unicode from inputs is a basic mitigation that should be standard.
• Test prompt injections with realistic context windows. Sparse test calendars produced false confidence. Real employee calendars dilute injections with legitimate content; this must be modeled in efficacy testing.
• AI agents can serve as initial access vectors. Goose's developer shell tool call, combined with MCP integrations that ingest untrusted external content, created a pathway from an external attacker account to code execution on an employee laptop — no vulnerability required, just social engineering and a novel injection.
• Social engineering remains effective even against security-aware targets. The spear-phishing campaign against the Goose development team succeeded because the cover story (external researchers reporting a bug) was plausible and the target behavior (running the recipe to reproduce the bug) was reasonable.

Slides Reference

Slides are available from the conference as 2026-04-04-D2-S1-13-57-Operation-Pale-Fire-How-We-Red-Teamed-.pdf. Key slides cover Goose's architecture and attack surface, the Google Calendar MCP attack chain, the invisible Unicode technique, the recipe system exploitation, the revised spear-phishing campaign, the payload design (modeled on a real-world infostealer), and the mitigation/detection stack (Unicode stripping, recipe transparency, Project Bashcat, command allowlisting). The Block engineering blog at engineering.block.xyz contains additional red team write-ups.