Operationalizing the Motivated Intruder: A Codebook-Guided Inference Framework for Semantic Input Privacy in LLMs

Michael Maximilian Grötzner (Paper author · FernUniversität in Hagen), Pascal Tippe (Paper author · FernUniversität in Hagen)

Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 2 · Moose

Overview

The proliferation of large language models (LLMs) has introduced a novel and complex challenge in the realm of data privacy: how to protect sensitive user input while still leveraging the powerful capabilities of these models. Users are often encouraged to provide extensive context to LLMs to achieve optimal results, leading to what the speakers term the "trust trap"—a scenario where the desire for better outputs inadvertently compromises privacy. This talk, presented by Michael Maximilian Grötzner at PETS, outlines a groundbreaking approach to addressing this issue through a Codebook-Guided Inference Framework for Semantic Input Privacy. The core innovation lies in operationalizing abstract legal norms, such as those found in GDPR and the German Trade Secret Protection Act, into an auditable and executable Codebook that guides a privacy gateway.

Watch on YouTube · Slides

Visual summary for Operationalizing the Motivated Intruder: A Codebook-Guided Inference Framework for Semantic Input Privacy in LLMs by Michael Maximilian Grötzner, Pascal Tippe
Visual summary for Operationalizing the Motivated Intruder: A Codebook-Guided Inference Framework for Semantic Input Privacy in LLMs by Michael Maximilian Grötzner, Pascal Tippe

Key moments

  1. 0:15 The 'trust trap' and semantic leakage problem
  2. 2:00 Introducing the Codebook-guided Privacy Gateway
  3. 2:30 The four-step sanitization cascade explained
  4. 4:15 Core research questions and legal foundations
  5. 6:00 Measuring sanitization efficiency and utility loss
  6. 7:00 Gateway achieves superior protection compared to Presidio
  7. 7:45 Analyzing the privacy-utility trade-off and latency
  8. 8:30 Conclusion: Semantic reasoning mandatory; future challenges

Operationalizing the Motivated Intruder: A Codebook-Guided Inference Framework for Semantic Input Privacy in LLMs

Speakers: Michael Maximilian Grötzner (Paper author, FernUniversität in Hagen); Pascal Tippe (Paper author, FernUniversität in Hagen)

Conference: PETS

YouTube: https://www.youtube.com/watch?v=PgLkv4iLHk

Overview

The proliferation of large language models (LLMs) has introduced a novel and complex challenge in the realm of data privacy: how to protect sensitive user input while still leveraging the powerful capabilities of these models. Users are often encouraged to provide extensive context to LLMs to achieve optimal results, leading to what the speakers term the "trust trap"—a scenario where the desire for better outputs inadvertently compromises privacy. This talk, presented by Michael Maximilian Grötzner at PETS, outlines a groundbreaking approach to addressing this issue through a Codebook-Guided Inference Framework for Semantic Input Privacy. The core innovation lies in operationalizing abstract legal norms, such as those found in GDPR and the German Trade Secret Protection Act, into an auditable and executable Codebook that guides a privacy gateway.

Traditional privacy-enhancing techniques, like synthetic pattern matching and differential privacy, fall short in mitigating the nuanced problem of semantic leakage, where individually benign pieces of information, when aggregated, can reveal sensitive insights. This research introduces a sophisticated, multi-stage sanitization cascade designed to detect, substitute, generalize, and redact sensitive information from user prompts before they reach the LLM. By integrating legal compliance directly into the technical framework, Grötzner and Tippe propose a robust solution against a motivated intruder who could exploit aggregated quasi-identifiers. This work is crucial for developers and organizations deploying LLMs, offering a path toward achieving robust input privacy without sacrificing the utility that makes these models so valuable.

The talk delves into the architecture of their Privacy Gateway, detailing its four-step sanitization process and the rigorous evaluation methodology used to assess its effectiveness. While demonstrating superior protection against semantic leakage compared to existing methods, the authors also transparently address the inherent trade-offs between privacy and utility, particularly regarding computational overhead and inference latency. This comprehensive analysis provides a critical perspective on the challenges and future directions for safeguarding user data in the age of advanced AI.

Background

▶ Watch: The 'trust trap' and semantic leakage problem (0:15)

The pervasive integration of large language models into daily applications has ushered in a new era of human-computer interaction, characterized by intuitive conversational interfaces. While these interfaces enhance user experience, they simultaneously create a significant privacy vulnerability, which the authors aptly identify as the "trust trap." Users are often motivated to provide extensive, detailed context to LLMs to elicit more accurate and relevant responses. This eagerness to maximize utility inadvertently leads users to disclose sensitive information, assuming—often incorrectly—that the underlying AI mechanisms inherently protect their data. The problem is exacerbated by the fact that end-users currently lack granular control over the internal privacy mechanisms of upstream AI services, and hosting large, powerful models locally remains cost-prohibitive for most.

Existing approaches to prompt sanitization have proven inadequate for the complexities of semantic privacy. Synthetic pattern matching, exemplified by tools like Microsoft Presidio, relies on predefined patterns or named entity recognition to identify and redact sensitive data. However, this method is fundamentally context-blind; it struggles to detect semantic leakage where sensitive information is not explicitly stated but can be inferred from the aggregation of individually innocuous pieces of data. For instance, knowing a user's location, profession, and recent online activity might seem benign in isolation, but together, these quasi-identifiers can uniquely identify an individual or reveal sensitive personal attributes.

Differential privacy (DP) offers a stronger theoretical guarantee of privacy by adding noise to data, making it difficult to infer information about any single individual. While DP can provide a baseline level of privacy, it typically operates at the training data level or on aggregate statistics. It fails to mitigate inference-time leakage, meaning sensitive information embedded directly within a user's prompt, which is essential for the LLM's task, can still be implicitly leaked or inferred by an attacker during the interaction. The core threat model considered in this research is the motivated intruder. This adversary is not merely looking for direct identifiers but actively uses open-source intelligence (OSINT) to correlate and cross-reference seemingly disparate quasi-identifiers present in leaked or inferred prompt data. By aggregating these subtle clues, the motivated intruder can reconstruct sensitive profiles or even identify the author of a prompt, thereby compromising privacy.

The challenge lies in the nature of semantic leakage itself, where sensitive information transcends standardized formats. It's not about explicit credit card numbers or social security codes (though those are covered), but rather the aggregation of contextual information that, when combined, becomes sensitive. To address this, the proposed framework is firmly grounded in established legal principles. Specifically, it operationalizes European General Data Protection Regulation (GDPR) Articles 4 and 9, which define personal identifiable information (PII) and highly sensitive personal information (e.g., medical data), respectively. Additionally, it incorporates the German Trade Secret Protection Act to ensure the confidentiality of proprietary company information, thereby providing a comprehensive legal and ethical foundation for its privacy mechanisms. This dual focus on technical robustness and legal compliance underscores the gravity and complexity of the problem being tackled.

Key Findings

▶ Watch: The four-step sanitization cascade explained (2:30)

The research presented by Grötzner and Tippe yields several critical findings that underscore both the effectiveness and the inherent challenges of achieving robust semantic input privacy in LLMs. Their Codebook-guided Privacy Gateway demonstrated a significant leap in protection compared to traditional methods, while also highlighting the computational trade-offs involved.

Firstly, the most striking finding concerns the superior protection against semantic leakage offered by their Privacy Gateway. The framework successfully redacted over 90% of semantic information deemed sensitive within the evaluated prompts. This contrasts sharply with Microsoft Presidio, a leading named entity recognition framework used for synthetic pattern matching, which failed heavily, catching only 5% of contextual secrets. This stark difference unequivocally demonstrates that context-blind approaches are insufficient for mitigating semantic leakage, validating the necessity of a semantic reasoning-based privacy solution.

Secondly, the evaluation methodology itself proved highly reliable. The process of annotating prompts and sanitized prompts with a human expert and an LLM ensemble, with disagreements resolved by a second human expert, yielded a high degree of consistency. The Krippendorff's Alpha for interrater reliability, a statistical measure of agreement between raters, was calculated to be between 0.7 and 0.8. This figure is considered on par with human-to-human rate agreement, indicating that the LLM ensemble, guided by the codebook, could effectively assess sensitive information detection with a reliability comparable to human judgment. This is a crucial finding, suggesting the potential for scalable, automated privacy assessment.

However, the research also clearly identified a significant privacy-utility trade-off. The sophisticated sanitization process, which involves deep semantic reasoning, is inherently compute-bound. On average, the Privacy Gateway introduced an inference latency of around 14 seconds per prompt. This delay is directly correlated with the depth of reasoning required to effectively sanitize the prompt. While 14 seconds might be acceptable for some asynchronous or less latency-sensitive applications, it poses a considerable challenge for real-time conversational interfaces where immediate responses are expected. This highlights a fundamental tension: greater privacy protection, especially at the semantic level, demands more computational resources and time.

Furthermore, the authors observed a semantic coupling limit. This refers to scenarios where specific contexts within a prompt are so intricately linked that sanitizing them sufficiently to ensure privacy makes it mutually exclusive to then receive a good, usable answer from the LLM. In essence, some sensitive information is so integral to the prompt's core intent that its removal or generalization renders the prompt meaningless or incapable of eliciting a useful response. This finding suggests that while the gateway can offer robust protection, there might be inherent limits to how much privacy can be achieved without fundamentally altering the utility for certain highly sensitive or context-dependent requests.

In summary, the key findings confirm that semantic reasoning is mandatory for achieving robust input privacy in LLMs, demonstrating the practical efficacy of their Codebook-guided Privacy Gateway. Yet, it also brings to light the accompanying challenges of increased inference time and the existence of a semantic coupling limit, which will be critical areas for future research and optimization.

Technical Deep Dive

▶ Watch: Measuring sanitization efficiency and utility loss (6:00)

The core innovation presented in this talk is the Codebook-guided Privacy Gateway, an architectural framework designed to provide robust semantic input privacy for LLMs. This gateway is strategically positioned within the trusted domain, acting as an intermediary between the user and the LLM, ensuring that sensitive information is meticulously handled before reaching the model.

At the heart of this framework is the concept of a Codebook. A Codebook, in this context, is not merely a list of rules but a set of executable instructions that operationalize abstract legal norms. It translates high-level privacy regulations, such as those found in GDPR Articles 4 and 9 (covering personal identifiable information and sensitive personal information like medical data) and the German Trade Secret Protection Act (for confidential company information), into concrete, actionable steps that both the LLM and the user can understand and follow. This approach replaces the opaque model weights of the LLM with an auditable and transparent privacy mechanism, giving users and organizations greater control and visibility over how sensitive data is managed.

The Privacy Gateway employs a strict four-step sanitization cascade for every incoming prompt:

  1. Goal-aware Triage: This initial step focuses on efficiency and relevance. The gateway analyzes the prompt to identify and delete any information that adds no value to the prompt's intended goal. This might include superfluous details, tangential remarks, or context that is not strictly necessary for the LLM to generate a good answer. By removing non-essential data early, the subsequent sanitization steps have less information to process, potentially reducing computational overhead and minimizing the risk of accidental leakage.
  1. Substitution: Following triage, the gateway targets direct identifiers. This step involves replacing specific, explicit identifiers within the prompt with generic placeholders or anonymized equivalents. Examples include project names, specific product codes, company names, or individual names that directly point to a specific entity. The goal here is to break direct links to sensitive entities without necessarily losing the semantic meaning required for the LLM's task. For instance, "Project Alpha" might become "Internal Project X."
  1. Generalization: This is a more nuanced step that addresses quasi-identifiers. Instead of direct replacement, generalization involves finding hypernyms (more general terms) for these identifiers. For example, if a prompt mentions a specific rare medical condition, the gateway might generalize it to "a medical condition" or "a chronic illness." Similarly, a specific job title like "Senior Cloud Security Architect at Acme Corp" might be generalized to "IT professional" or "security expert." This step aims to obscure specific details that, while not directly identifying on their own, could contribute to re-identification when aggregated, all while preserving enough semantic context for the LLM to understand the underlying request.
  1. Redaction: The final sanitization step is a fallback for information that simply cannot be sanitized through triage, substitution, or generalization without completely destroying the prompt's utility or risking severe leakage. In these cases, the information is redacted entirely. This might apply to highly specific details that are both sensitive and inseparable from their context, where any attempt at generalization would still leave too much room for inference.

After the four-step cascade, a crucial recursive check is performed. The sanitized prompt is re-evaluated against the Codebook to ensure that no residual leakage remains. If any sensitive information is detected, the entire sanitization cascade is run through again until the prompt meets the privacy requirements defined by the Codebook. This iterative process ensures a high degree of confidence in the privacy of the final input.

The entire framework operates on the principle of semantic reasoning, meaning it understands the context and meaning of the information rather than just performing pattern matching. This capability is what allows it to effectively counter the threat of semantic leakage and the motivated intruder, moving beyond surface-level sanitization to address the deeper privacy implications of conversational AI.

Demo / Proof of Concept

▶ Watch: Gateway achieves superior protection compared to Presidio (7:00)

While the talk did not feature a live, interactive demonstration in the traditional sense, the authors presented a rigorous and comprehensive evaluation framework that served as the proof of concept for their Codebook-guided Privacy Gateway. This framework was meticulously designed to objectively measure the gateway's effectiveness in detecting and sanitizing sensitive information, as well as quantifying the associated privacy-utility trade-off.

The evaluation commenced with a carefully curated dataset comprising 227 prompts. This dataset was strategically split into a development set, used for refining the methodology and building the Codebooks, and a separate test set, used for the final, unbiased evaluation of the gateway's performance. To ensure a high-fidelity proxy for real-world LLM interactions, they utilized Gemini 2.5 Pro for generating responses to both original and sanitized prompts. For comparative analysis, the Privacy Gateway was benchmarked against Microsoft Presidio, a well-known named entity recognition (NER) framework, which served as a representative of synthetic pattern matching approaches to prompt sanitization.

A critical component of their proof of concept involved the development of two distinct Codebooks: one for measuring the sensitive information contained in the prompts (both before and after sanitization) and another for quantifying the utility loss resulting from the sanitization process. These Codebooks were essential for establishing objective criteria for evaluation.

To measure sanitization efficiency, the following process was implemented:

  1. Each original prompt and its corresponding sanitized version were independently annotated.
  2. Annotation was performed by a human expert and an LLM ensemble, both guided by the sensitive information Codebook.
  3. In cases of disagreement between the human expert and the LLM ensemble, a second human expert was brought in to resolve the discrepancy. This multi-layered approach ensured high accuracy and mitigated individual biases in identifying sensitive content. This process allowed the researchers to objectively quantify how much sensitive information was present in the prompt initially and how much remained after the Privacy Gateway's intervention.

To objectively measure utility loss, a similarly robust process was devised:

  1. For each prompt, the responses from the LLM (Gemini 2.5 Pro) to the original prompt and the sanitized prompt were compared.
  2. Two independent human experts performed the initial coding using the utility loss Codebook.
  3. A third human expert was then engaged to resolve any disagreements between the first two. Notably, the researchers explicitly chose not to use an LLM for the final resolution in this phase, as they "recognized that the LLM in this case preferred sometimes the first answer over the second answer," indicating an ordering bias. This decision highlights the meticulous attention to detail and methodological rigor employed to ensure the objectivity of the utility assessment.

Through this comprehensive evaluation framework, the researchers sought to answer three core research questions:

  1. Reliability of Detection: How reliably can the gateway detect sensitive information?
  2. Quality of Sanitization: How effectively can it sanitize the information?
  3. Privacy-Utility Trade-off: What is the balance between privacy protection and the usability of the LLM's answers to sanitized prompts?

The results of this rigorous evaluation, as detailed in the "Key Findings" section, provided empirical evidence for the superior protection offered by the Codebook-guided Privacy Gateway, while also clearly delineating the practical implications of its computational demands and the semantic coupling limits. This detailed methodology serves as a compelling proof of concept for the feasibility and effectiveness of their proposed framework.

Defensive Implications

▶ Watch: Conclusion: Semantic reasoning mandatory; future challenges (8:30)

The findings presented in "Operationalizing the Motivated Intruder" carry significant implications for developers, security professionals, and organizations grappling with the challenge of LLM input privacy. The research provides a clear roadmap for enhancing defensive postures against semantic leakage and the sophisticated tactics of a motivated intruder.

First and foremost, the talk underscores the critical need to move beyond simplistic pattern matching for prompt sanitization. Defenders must recognize that context-blind approaches are fundamentally insufficient for protecting against semantic leakage, where sensitive information is inferred rather than directly stated. Organizations currently relying solely on regular expressions or basic named entity recognition tools (like Microsoft Presidio, which caught only 5% of contextual secrets in the evaluation) for LLM input privacy are operating with a false sense of security. The Codebook-guided Privacy Gateway demonstrates that semantic reasoning is mandatory for robust protection.

Organizations should consider implementing or developing their own privacy gateway architectures that operate within their trusted domain. This gateway should act as a mandatory intermediary for all LLM interactions, scrutinizing user prompts before they reach the upstream model. The concept of the Codebook is a powerful defensive tool, offering a way to operationalize abstract legal norms (like GDPR and trade secret laws) into executable, auditable instructions. This provides a transparent and legally compliant framework for data handling, enabling organizations to demonstrate due diligence and accountability in their privacy practices. Developers should invest in creating or integrating such Codebooks, ensuring they are regularly updated to reflect evolving legal landscapes and threat models.

The proposed four-step sanitization cascade (goal-aware triage, substitution, generalization, and redaction) offers a practical blueprint for implementing granular privacy controls. Defenders should integrate these stages into their data processing pipelines, customizing them to the specific types of sensitive information their users might provide and the regulatory requirements they face. The recursive check after sanitization is also a crucial defensive measure, ensuring that no residual leakage escapes the gateway.

However, defenders must also be acutely aware of the privacy-utility trade-off. The observed inference latency of around 14 seconds per prompt and the compute-bound nature of semantic reasoning mean that implementing such a robust gateway will introduce overhead. Organizations must carefully evaluate their use cases and user expectations. For applications requiring real-time conversational responses, this latency could be a significant barrier. Future defensive strategies will need to focus on optimizing these semantic reasoning engines, perhaps through techniques like model distillation, hardware acceleration, or exploring smaller, more efficient privacy models.

Furthermore, the semantic coupling limit highlights a fundamental challenge: some information is so intrinsically sensitive and critical to the prompt's utility that its effective sanitization may render the prompt useless. Defenders must communicate this limitation to users, perhaps by providing clear guidelines on what kind of information cannot be safely provided to an LLM, even with a privacy gateway in place. This emphasizes the need for user education and responsible data input practices.

In conclusion, the defensive implications are clear: a proactive, semantic reasoning-based approach to LLM input privacy is no longer optional but essential. Organizations must invest in sophisticated privacy gateway technologies guided by auditable Codebooks, accept the inherent privacy-utility trade-offs, and continuously work towards optimizing the efficiency of these critical security layers to safeguard user data in the AI era.

Key Takeaways

  • Semantic Leakage is a Critical Threat: Traditional prompt sanitization methods like synthetic pattern matching are context-blind and fail to protect against semantic leakage, where sensitive information can be inferred from individually benign quasi-identifiers aggregated by a motivated intruder.
  • The "Trust Trap" Demands New Solutions: Users are incentivized to provide extensive context to LLMs, leading to a "trust trap" where utility-seeking behavior inadvertently compromises privacy due to a lack of control over internal AI mechanisms.
  • Codebook-Guided Privacy Gateways Offer Superior Protection: The proposed Codebook-guided Privacy Gateway operationalizes abstract legal norms (e.g., GDPR Articles 4 & 9, German Trade Secret Protection Act) into executable instructions, achieving over 90% redaction of semantic information compared to 5% by Microsoft Presidio.
  • Multi-Stage Sanitization Cascade is Effective: A strict four-step cascade—goal-aware triage, substitution of direct identifiers, generalization of quasi-identifiers, and final redaction—combined with a recursive check, is highly effective for robust input privacy.
  • Significant Privacy-Utility Trade-off Exists: Achieving robust semantic privacy is compute-bound, leading to an average inference latency of 14 seconds per prompt. This highlights a critical challenge for real-time applications and the need for future optimization.
  • Semantic Reasoning is Non-Negotiable: For truly robust LLM input privacy, semantic reasoning is mandatory. Defenders must move beyond surface-level pattern matching and adopt deeper, context-aware mechanisms, despite the associated computational costs and potential semantic coupling limits.

About the Speaker(s)

The research presented in this talk was authored by Michael Maximilian Grötzner and Pascal Tippe, both affiliated with FernUniversität in Hagen. Michael Maximilian Grötzner was the presenter of the talk at the PETS conference. Their work focuses on advanced privacy solutions for large language models, addressing the complex interplay between user utility, data security, and legal compliance in the evolving landscape of AI. Their expertise lies in developing frameworks that translate abstract privacy regulations into actionable, technical mechanisms for protecting sensitive user input.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

They built a semantic privacy gateway to sanitize LLM prompts because regex-based tools like Microsoft Presidio are completely blind to context. The real win here is their brutal honesty: they admit this adds 14 seconds of latency and often destroys the prompt's utility to save its privacy. Solid, rigorous work that proves exactly why the 'trust trap' is so hard to engineer around.

Heather Calloway (CISO) — SOLID

This research proves what practitioners already suspected: traditional pattern-matching DLP is effectively useless for protecting LLM inputs. The authors demonstrate a semantic gateway that actually works for GDPR and trade secret compliance, but it carries a 14-second latency penalty that forces a hard conversation about the cost of AI privacy.

→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026

All talks from Privacy Enhancing Technologies Symposium (PETS) 2026