Securing Workspace GenAI at Google Speed: Surviving the Perfect Storm

Securing Workspace GenAI at Google Speed: Surviving the Perfect Storm

Speaker: Nicolas Lidzborski (Nico), Principal Engineer, Google Workspace Security

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 4, 2026, San Francisco

Watch: YouTube — Full Talk

Reading time: ~9 min

TL;DR

The generative AI era has collapsed the traditional distinction between code and data — making every token in an LLM's context a potential instruction and rendering reactive filtering fundamentally futile. Nicolas Lidzborski, a 25-year security veteran who has spent three years securing Gemini in Google Workspace, presents a four-layer structural blueprint for building AI defenses that actually hold: low-risk input preparation, context hardening, deterministic orchestration, and output sanitization. The talk ends with a concrete demonstration of why the cat-and-mouse approach to prompt filtering is a game defenders cannot win.

Introduction

Nicolas Lidzborski — "Nico" — opens his [un]prompted 2026 talk with a striking framing: the industry is in a "perfect storm" where the security boundaries that have protected applications for decades are simply no longer enough. And unlike most speakers who reference this idea in passing, Lidzborski has spent three years actively trying to build defenses for one of the most broadly deployed AI productivity platforms in the world — Google Workspace.

"I want to share lessons so people aren't repeating our mistakes," he told the audience. Over a 25-minute talk, he covers the anatomy of the threat, explains why filtering is a losing strategy, proposes a concrete four-layer architectural blueprint, and closes with what systemic resilience actually looks like at Google's scale.

The perspective is unusually practical. Lidzborski is not describing theoretical vulnerabilities — he is describing attacks that have hit Google Workspace and defenses that have been tested against real adversaries.

▶ Watch: Introduction (00:00)

The Core Threat: Prompt as Code

Traditional application security is built on a clean separation between code and data. SQL injection is exploitable because untrusted data gets mixed with executable SQL; the fix — parameterized queries — re-establishes that separation. Buffer overflows are exploitable because data written to memory can overwrite code; the fix is enforcing the boundary.

In LLMs, that boundary does not exist. The system prompt, the user message, and the external data ingested from a calendar invite or email all arrive in the model as a single, contiguous token stream. There is no NX bit for the context window. Every token is a potential instruction. As Lidzborski put it: "There is no out-of-band way to tell the model that these 500 tokens are just data — do not execute them."

This creates two synergistic problems:

The semantic shift. Traditional application security operates on deterministic parsing — SQL has a recognizable shape. Generative AI operates on intent interpretation. Attackers can use linguistic techniques (persuasion, role-playing, persona adoption) to bypass filters without needing to exploit any syntactic vulnerability. Software engineers often ask why prompt injection can't be blocked "like SQL injection" — and the honest answer is that there is no equivalent of parameterized queries for natural language.

The collapsed control plane. In classical computing, the code that executes is separate from the data it acts on. In an LLM, the instruction and the data are in the same stream. Untrusted external data can overwrite instructions — and that is the definition of prompt injection.

▶ Watch: Prompt as code (02:00)

Anatomy of the Attack: Indirect Prompt Injection and the Lethal Trifecta

Lidzborski describes indirect prompt injection as the primary threat for productivity environments like Google Workspace. The defining characteristics:

• Hidden — the malicious payload is buried in external untrusted content (a calendar invite, an email body, a shared document), not in the user's direct prompt
• Zero-click — the attack triggers when the user performs a completely benign action, like asking an AI to "summarize my day"
• Scalable — a single malicious document or invite can propagate silently across thousands of users who interact with it

He references a concrete real-world demonstration: a malicious payload hidden inside a Gmail message, invisible to the user. When the AI agent performs the benign task of email summarization, it ingests the hidden text, which overrides its instructions and executes unauthorized actions. "The user sees nothing. The attacker wins."

The second major risk is Markdown exfiltration — a rendering attack where an LLM is prompted to generate Markdown containing images or links hosted on attacker-controlled servers. Sensitive data gets silently appended to the image URL as a query parameter. When the rendered output triggers a request to fetch the image, the data arrives in the attacker's server logs. The user sees only a broken image icon.

The third risk is rogue actions in agentic environments, driven by three sub-problems: the agency gap (non-deterministic agent reasoning leading to accidental harmful actions), orchestration hijacking (compromised orchestration layer manipulated via indirect prompt injection, including dormant triggers that fire long after the initial injection), and the confused deputy problem (agents granted permissions exceeding what the user actually authorized).

Lidzborski invokes Simon Willison's "lethal trifecta" — sensitive private data access, continuous untrusted content exposure, capability to execute external commands — to describe the convergence that makes these risks so serious in productivity platforms.

The attack demonstration from Black Hat (the "Invitation Is All You Need" paper by Ben Nassi, Stav Cohen, and co-authors) makes this concrete: an attacker sends a Google Calendar invite with a hidden payload. When the user asks "What's on my schedule today?" — one of the most benign queries imaginable — the agent ingests the invite, follows the hidden instructions, and can move laterally or exfiltrate data. In Google's case, the data exfiltration was protected — but the team had not anticipated that "home actions" could also be triggered: controlling smart lights, curtains, thermostats. No zero-day. No malware. No link clicked.

▶ Watch: The calendar invite attack (10:00)

Why Filtering Is a Losing Game

Lidzborski is explicit that reactive filtering — the instinctive first response to prompt injection — is fundamentally inadequate. He outlines three reasons:

Static filters (block lists, regex) operate on syntax and are defeated trivially by encoding. ROT13, hex encoding, Base64 — the LLM understands any of them without special instructions. The moment the data is encoded, the pattern breaks.

ML classifiers sound more sophisticated but still lack semantic depth. Attackers bypass probabilistic models by swapping synonyms, translating payloads into low-resource languages, or using adversarial prefixes that nudge the model's confidence just enough to slip through. "You only need one bypass to get through."

Text-only filters are completely blind to multimodal attacks. Instructions hidden in image metadata or encoded using OCR-evasion techniques bypass standard text processing pipelines entirely.

The use of a secondary LLM as a "judge" — reviewing outputs before they reach the user — has a specific and critical flaw: recursive prompt injection. The judge reviews the same untrusted data as the primary model. Because the attacker and the judge share the same semantic interface, payloads can include instructions specifically aimed at the judge — what Lidzborski calls "semantic gaslighting": instructing the judge to evaluate the following content as safe, even if it contains execution commands, because it's for research purposes.

"Reactive guardrails and pattern matching are inherently inadequate. You do not want to play cat-and-mouse with threat actors who are constantly developing new evasion techniques. You will always be behind."

▶ Watch: Why filtering fails (10:00)

Architecting the Fortress: A Four-Layer Structural Blueprint

The core of the talk is a four-layer defense architecture that shifts from reactive filtering to structural protection.

Layer 1: Low-risk input preparation. Before content ever reaches the LLM, strip everything that isn't visible to the end user. Hidden HTML elements, invisible characters, non-rendered metadata — none of it should be passed to the model. Apply abuse signal filtering: if an email is already classified as phishing or spam, there's no reason to expose the LLM to it. Use data provenance tracking to enforce policy throughout the system — user risk scores, affinity signals, prompt injection classification — so the system always knows where content originated.

Layer 2: Context hardening via prompt delimitation. Use sentinel tokens to mark trusted system instructions versus untrusted external data within the context window. Combine with adversarial training to harden the model against linguistic manipulation. This doesn't eliminate the risk but meaningfully raises the bar.

Layer 3: Deterministic orchestration. The orchestration layer — the code that decides what the agent does and which tools it calls — should enforce security policy deterministically, not probabilistically. Google uses a finite state machine tracking the risk level of context: as the agent ingests content from untrusted sources, the state machine progressively restricts downstream capabilities. After ingesting external data, web requests should be denied. Mutations and data sharing require human confirmation. These are deterministic rules applied at every orchestration step, not just at the initial prompt.

Layer 4: Output sanitization and link grounding. LLM outputs arrive as Markdown that gets rendered as HTML. Google parses and scrubs that Markdown to eliminate anomalies: unwanted image embeds, suspicious link protocols. Dynamic URL classification checks all URLs in the output against Safe Browsing data. Hallucinated links — URLs the LLM invented without grounding in the source material — are scrubbed entirely.

▶ Watch: The four-layer blueprint (14:00)

Systemic Resilience: Continuous Testing and User Feedback Loops

Defense architecture alone is not enough — it requires continuous validation. Google runs an automated testing framework that systematically subjects systems to all known classes of agentic attacks, specifically designed to catch regressions (ensuring defenses built last month aren't broken by a feature deployed this week). The probabilistic nature of LLMs matters here: "Sometimes you try the same attack ten times and one out of ten it will work. You won't get rid of that problem."

Lidzborski also emphasizes red teaming as a collaborative function — dedicated red teams finding zero-day vectors, co-engineering teams implementing structural defenses, and external researchers engaged through vulnerability reward programs (VRP) and events like BugSwap.

For high-stakes irreversible actions, the pattern is plan, validate, execute: the agent generates a plan, shows it to the user for confirmation, then executes. The UX challenge of review fatigue and rubber-stamping is acknowledged as an unsolved problem requiring ongoing research.

The final pillar is a feedback loop with users: empowering users to report AI hallucinations and suspected prompt injections, incorporating those reports into detection models, and continuously adjusting sensitivity. "Security is not a line but a loop."

▶ Watch: Systemic resilience (18:01)

Notable Quotes

"In the GenAI world, the prompt is code. Every single token in the input stream is a potential instruction." — ▶ 02:00

"You do not want to play cat-and-mouse with threat actors who are constantly developing new evasion techniques. You will always be behind." — ▶ 10:00

"No zero-day exploited. No malware installed. No link clicked. Real-world consequences." — ▶ 10:00

"Agent vulnerabilities live in language and context. Traditional vulnerabilities live in poorly sanitized strings or broken authentication checks." — ▶ 12:00

"We need to build systemic semantic sandboxes with defense in depth. The cat-and-mouse game of filtering is over." — ▶ 20:01

Key Takeaways

• The control plane has collapsed. In LLMs, instructions and data share the same token stream. This is not a fixable bug — it is the architecture of these systems, and all defenses must be designed with this in mind.
• Filtering is not a viable primary defense. Static filters, ML classifiers, and even LLM judges are all bypassable through semantic manipulation. Defenders who rely on filtering will always be behind.
• Structural defense requires four layers. Input preparation, context hardening, deterministic orchestration, and output sanitization must all be implemented together. Each layer provides defense that the others cannot.
• Dormant triggers are a real and underappreciated threat. Prompt injections don't need to execute immediately — they can be stored in databases and triggered by events that occur long after the initial injection.
• Security is a continuous loop, not a one-time deployment. Automated regression testing, user feedback integration, and ongoing red teaming are operational requirements for AI security, not optional enhancements.

Slides Reference

Slides are available from the conference as 2026-04-04-D2-S1-13-31-Securing-Workspace-GenAI-at-Google-Spe.pdf. Key slides cover the anatomy of the threat (semantic shift, prompt-as-code), the indirect prompt injection attack structure, Markdown exfiltration mechanics, the lethal trifecta framework, the four-layer defense blueprint, and the systemic resilience pillars (automated testing, plan-validate-execute pattern, user feedback loops).