Personalizing Agent Privacy Decisions via Logical Entailment

James Flemings (Paper author · University of Southern California), Ren Yi (Paper author · Google), Octavian Suciu (Paper author · Google), Kassem Fawaz (Paper author · University of Wisconsin--Madison), Murali Annavaram (Paper author · University of Southern California), Marco Gruteser (Paper author · Google)

Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 3 · Caribou

Overview

As large language models (LLMs) evolve from static text generators into autonomous agents capable of performing complex tasks on behalf of users, they increasingly encounter and must handle sensitive personal information. This presents a critical challenge: how can these agents make appropriate data sharing decisions without overwhelming users with constant privacy prompts, a phenomenon known as "privacy fatigue"? The talk "Personalizing Agent Privacy Decisions via Logical Entailment" by James Flemings and his co-authors introduces Ariel (Agentic Reasoning via Individualized Entailment Logic), a novel framework designed to address this dilemma by automating routine privacy decisions while preserving user control and providing transparency.

Watch on YouTube · Slides

Visual summary for Personalizing Agent Privacy Decisions via Logical Entailment by James Flemings, Ren Yi, Octavian Suciu, Kassem Fawaz, Murali Annavaram, Marco Gruteser
Visual summary for Personalizing Agent Privacy Decisions via Logical Entailment by James Flemings, Ren Yi, Octavian Suciu, Kassem Fawaz, Murali Annavaram, Marco Gruteser

Key moments

  1. 0:20 LLM agents, sensitive data, and privacy fatigue
  2. 2:05 Limitations of purely relying on LLM privacy norms
  3. 3:35 Framing privacy decisions as a logical entailment problem
  4. 5:00 Using ontologies to capture data sensitivity for entailment
  5. 6:35 Ariel system overview: offline ontology generation, online reasoning
  6. 8:50 Ariel improves personalized privacy decisions over LLMs
  7. 9:20 LLMs struggle to abstain when uncertain about privacy decisions

Personalizing Agent Privacy Decisions via Logical Entailment

Speakers: James Flemings (Paper author, University of Southern California); Ren Yi (Paper author, Google); Octavian Suciu (Paper author, Google); Kassem Fawaz (Paper author, University of Wisconsin--Madison); Murali Annavaram (Paper author, University of Southern California); Marco Gruteser (Paper author, Google)

Conference: PETS

YouTube: https://www.youtube.com/watch?v=azHZ2O1vAIY

Overview

As large language models (LLMs) evolve from static text generators into autonomous agents capable of performing complex tasks on behalf of users, they increasingly encounter and must handle sensitive personal information. This presents a critical challenge: how can these agents make appropriate data sharing decisions without overwhelming users with constant privacy prompts, a phenomenon known as "privacy fatigue"? The talk "Personalizing Agent Privacy Decisions via Logical Entailment" by James Flemings and his co-authors introduces Ariel (Agentic Reasoning via Individualized Entailment Logic), a novel framework designed to address this dilemma by automating routine privacy decisions while preserving user control and providing transparency.

The core problem Ariel tackles is the need for LLM agents to make nuanced, personalized data sharing judgments. For instance, an agent booking a restaurant reservation might need to determine if it's appropriate to share a user's credit card number with a third-party restaurant. While constantly asking the user is an option, it leads to fatigue. Prior approaches in the Internet of Things (IoT) domain, such as classical machine learning or rule-based systems, fall short in the dynamic, diverse data landscape of LLM agents, requiring continuous retraining or redesign. Although LLMs exhibit some awareness of privacy norms, relying solely on this awareness is problematic due to their inability to capture user-specific nuances, their propensity for hallucination, and their inherent black-box nature.

Ariel proposes a solution by reframing privacy decision-making as a logical entailment problem. By leveraging a user's past privacy decisions and a structured understanding of data context, Ariel aims to infer appropriate actions for new, similar data sharing requests. This approach seeks to align agent decisions with user preferences, provide interpretable reasoning for each decision, and minimize privacy fatigue by automating predictable choices while escalating ambiguous or complex situations back to the user for direct input. The work represents a significant step towards building more trustworthy and user-centric AI agents that can intelligently manage sensitive information.

Background

▶ Watch: LLM agents, sensitive data, and privacy fatigue (0:20)

The challenge of enabling automated systems to make privacy-preserving data sharing decisions is both a classical and an emerging problem. Before the advent of sophisticated AI agents, this issue was extensively studied in the context of Internet of Things (IoT) devices, particularly smart home personal assistants like Amazon Alexa. Early solutions often involved classical machine learning (ML) based approaches, such as logistic regression, or rule-based systems to infer user privacy preferences. These methods typically relied on predefined categories of data and recipients, learning patterns from user interactions to automate decisions.

However, the landscape has significantly shifted with the rise of large language model (LLM) agents. These agents operate in much broader and more dynamic environments, handling a vast array of unseen data types and interacting with an ever-expanding set of recipients. The limitations of classical approaches become apparent here: they would require constant retraining or extensive redesign to cope with the novel contexts LLM agents encounter, making them impractical.

A recent line of research has shown that LLMs possess an inherent awareness of privacy norms, suggesting their potential in automating privacy decisions. This is a promising development, as LLMs can understand and reason about complex textual descriptions of data sharing scenarios. Nevertheless, this talk identifies two critical limitations of purely relying on LLMs' privacy norm awareness:

  1. Nuance and Variance: LLMs' general understanding of privacy norms often fails to capture the subtle nuances and individual variances in user privacy preferences. What one user considers acceptable, another might not, even for similar data.
  2. Hallucination and Black Box: LLMs are known to hallucinate, generating plausible but incorrect information. Furthermore, their decision-making processes are inherently black box, making it difficult for users to understand why a particular privacy decision was made, eroding trust.

These limitations inform three crucial design requirements for creating effective personal privacy agents:

  • Alignment: Every privacy decision made by the agent must align with the user's prior, explicit data sharing decisions. This ensures consistency and reflects the user's established preferences.
  • Interpretability: For every decision, the agent should be able to provide clear, interpretable reasoning. This allows users to review and understand the rationale behind the agent's actions, fostering trust and enabling correction if necessary.
  • User Agency: The system must minimize privacy fatigue by automating routine or easily inferable decisions. However, this automation should not come at the expense of cognitive offloading, meaning users must still feel in control. Complex or ambiguous decisions should be escalated to the user, ensuring they retain ultimate authority over sensitive data.

Ariel is designed to meet these requirements by introducing a structured, logical approach that complements the LLM's natural language understanding capabilities, moving beyond simple norm awareness to personalized, verifiable reasoning.

Key Findings

▶ Watch: Framing privacy decisions as a logical entailment problem (3:35)

The research presented in this talk highlights several significant findings regarding the effectiveness and design principles of Ariel in personalizing agent privacy decisions:

  1. Improved Performance over Pure LLM Approaches: Ariel consistently demonstrates a substantial performance improvement in making appropriate and inappropriate privacy judgments compared to purely LLM-based baselines. When evaluated using the F1 score for both appropriate and inappropriate decisions, Ariel outperforms In-Context Learning (ICL) and ICL with undetermined models. This indicates that Ariel's structured reasoning framework, which leverages logical entailment and ontologies, is more effective at capturing user-specific privacy preferences than relying solely on the LLM's inherent understanding.
  1. Accurate Identification and Escalation of Challenging Requests: A key strength of Ariel lies in its ability to accurately identify incoming requests that are ambiguous or challenging, and consequently, escalate them to the user. The analysis showed that for requests Ariel deemed "not entailed" (i.e., it couldn't confidently make a decision based on prior user input), the performance of the ICL baseline substantially decreased. This suggests that Ariel effectively acts as a filter, allowing the LLM to handle straightforward cases while delegating complex, high-stakes decisions to the human user, thereby fulfilling the user agency requirement and reducing the risk of erroneous automated decisions. This capability is crucial, as LLMs often struggle to "abstain" or output "undetermined" when they lack sufficient information for a privacy decision, making them prone to errors in uncertain scenarios.
  1. Insufficiency of LLM Privacy Norm Awareness Alone: The evaluation against a zero-shot baseline further underscored that relying solely on the privacy norm awareness of LLMs is insufficient for the problem of personalized privacy decision-making. The zero-shot performance was notably lower, reinforcing the argument that while LLMs understand general privacy concepts, they lack the specific, personalized context necessary to make aligned decisions for individual users. Ariel's approach of conditioning LLM-generated ontologies on user-specific prior decisions is therefore critical for achieving true personalization.
  1. Balancing Automation and User Control: Ariel successfully addresses the core tension between automating privacy decisions to combat privacy fatigue and ensuring users retain control over their data. By automating inferable decisions through logical entailment and escalating ambiguous ones, Ariel provides a framework that enhances efficiency without sacrificing user agency. The soup metric, which measures the number of appropriateness decisions made, implicitly highlights Ariel's ability to automate a significant portion of decisions while still allowing for user intervention when necessary.

In summary, Ariel provides a robust and interpretable method for personalizing agent privacy decisions, demonstrating that a hybrid approach combining LLMs with structured logical reasoning yields superior results compared to LLMs operating in isolation.

Technical Deep Dive

▶ Watch: Using ontologies to capture data sensitivity for entailment (5:00)

The foundational concept behind Ariel is to reframe privacy decision-making as a logical entailment problem. This means the agent asks: "Does a user's prior decision on a prior request logically entail the same decision for an incoming request?" This approach allows the system to infer new decisions based on established user preferences, providing a structured and interpretable mechanism.

To operationalize this, Ariel represents each data sharing request using five fields derived from contextual integrity. These fields provide a comprehensive context for understanding the nature of the data transmission:

  1. Data Type: The specific kind of information being shared (e.g., full Social Security Number, partial SSN, public profile).
  2. Data Subject: The individual to whom the data belongs (typically the user).
  3. Data Sender: The entity initiating the data transmission (e.g., the user's agent).
  4. Data Recipient: The entity receiving the data (e.g., a bank, a restaurant).
  5. Transmission Principle: The purpose or condition under which the data is being transmitted (e.g., "to open a checking account," "to complete a reservation").

Consider an illustrative example: A user previously deemed it "appropriate" for a bank to request their full Social Security Number (SSN) to open a checking account. Now, an incoming request asks for a partial SSN (e.g., the last four digits) for the same purpose and recipient. Intuitively, if the user was comfortable sharing a more sensitive piece of data (full SSN), they would likely be comfortable sharing a less sensitive, subset of that data (partial SSN) under identical circumstances. This is the essence of logical entailment in Ariel: if sharing more sensitive data was appropriate, then sharing less sensitive data in the same context should also be appropriate.

Ariel achieves this operationalization through the use of ontologies. In this context, an ontology is a structured hierarchy or graph for each of the five fields, encoding specific relationships relevant to privacy. For instance, the data type ontology captures data sensitivity. This ontology might define levels of sensitivity, such as:

  • L1: Least sensitive (e.g., publicly available information).
  • L2: Moderately sensitive (e.g., general location data).
  • L3: Sensitive (e.g., partial SSN, email address).
  • L4: Highly sensitive (e.g., full SSN, credit card number).

These levels are designed to be broad and abstract, allowing them to encompass a wide variety of data types.

The process involves the agent mapping the data type for both the prior and incoming requests to their corresponding sensitivity levels within the ontology. For example, a "full SSN" might map to L4, while a "partial SSN" maps to L3.

Once the requests are mapped, they are passed to an entailment logic block. This block performs a series of simple checks to determine if entailment holds. The specific check executed depends on the judgment of the prior request (e.g., "appropriate" or "inappropriate").

  • If the prior request was deemed "appropriate," the logic block might check if the data sensitivity of the prior request is greater than or equal to the sensitivity of the incoming request, given all other fields are identical or entail each other (e.g., recipient is the same, purpose is the same). In our SSN example, L4 (full SSN) is indeed larger than L3 (partial SSN), so the check holds.
  • If the prior request was deemed "inappropriate," the logic might check for the opposite, or for situations where a less sensitive prior request was rejected, implying the more sensitive incoming request should also be rejected.

If the entailment check holds, Ariel reuses the judgment of the prior request for the incoming request. If it does not hold, the incoming request is escalated to the user for a direct decision, fulfilling the user agency requirement.

The entire Ariel system operates in two distinct phases:

  1. Offline Phase: In this phase, Ariel generates the necessary ontologies. An LLM is conditioned on the user's decisions on prior requests. This is crucial for personalization; the LLM learns the user's specific privacy preferences from their historical choices and uses this understanding to construct or refine the field ontologies (e.g., the data sensitivity hierarchy for different data types, or relationships between recipients). This ensures the ontologies reflect the individual user's perspective rather than generic norms.
  1. Online Phase: During this phase, the agent receives an incoming data sharing request.
  • The agent first retrieves a relevant prior request from its memory.
  • It then maps the fields of both the incoming and prior requests to their corresponding levels or nodes within the pre-generated ontologies.
  • These mapped requests are then fed into the entailment logic block.
  • If entailment holds, the agent reuses the prior judgment.
  • If entailment does not hold, Ariel iterates through other relevant prior requests in memory, attempting to find another match that entails a decision.
  • If, after checking all relevant prior requests, no entailment is found, the incoming request is escalated to the user, ensuring that ambiguous or uninferable decisions are always handled by the human.

This two-phase approach, combining LLM-driven ontology generation with structured logical entailment, allows Ariel to make personalized, interpretable, and aligned privacy decisions while minimizing user fatigue.

Demo / Proof of Concept

▶ Watch: Ariel improves personalized privacy decisions over LLMs (8:50)

While the talk did not feature a live, interactive demonstration of the Ariel system, the effectiveness of its approach was rigorously evaluated using a comprehensive dataset and established metrics. The research utilized the smart home personal assistant dataset from Upadhyay et al. (2021). This dataset is a crucial component of the evaluation, as it provides a realistic context for privacy decisions involving autonomous agents.

The dataset itself originates from a user study that collected responses to various data sharing scenarios involving common smart home devices, such as Amazon Alexa. Each scenario within the dataset is meticulously mapped to the five contextual integrity fields that Ariel employs: data type, data subject, data sender, data recipient, and transmission principle. An example scenario might describe a smart thermostat requesting access to a user's calendar to optimize heating schedules, which would then be broken down into these five structured components.

For the evaluation, the researchers sampled 500 distinct users from the dataset. For each user, 70 data sharing responses were selected. Of these, 60 responses were designated as "prior requests," serving as the historical context of user decisions that Ariel would learn from and compare against. The remaining 10 responses were designated as "incoming requests," which Ariel had to make a privacy decision on.

The primary LLM model used for presenting results in the talk was Gemini 2.5 Pro, though the paper contains results for additional models. Ariel's performance was measured using two key metrics:

  • F1 score: This metric was used to evaluate the accuracy of Ariel's judgments for both "appropriate" and "inappropriate" decisions. A higher F1 score indicates better precision and recall in making correct privacy calls.
  • Soup metric: This metric quantified the number of "appropriateness" decisions made by Ariel. Importantly, if Ariel could not confidently make a judgment (i.e., no entailment was found), it would escalate the request to the user. These escalated requests were not evaluated by the F1 score, making the "soup metric" an indicator of how many decisions Ariel could automate successfully.

Ariel's performance was benchmarked against several baselines:

  • ICL (In-Context Learning): This baseline involved conditioning the LLM with the 60 prior requests for a given user and then asking it to make a privacy decision on the 10 incoming requests purely based on its in-context learning capabilities.
  • ICL with undetermined: This was an extension of the ICL baseline, giving the LLM the explicit option to output "undetermined" if it could not reasonably answer the decision, providing a more direct comparison to Ariel's escalation mechanism.
  • Zero-shot: This baseline represented a scenario where the LLM relied solely on its pre-trained privacy norm awareness without any user-specific prior examples, highlighting the insufficiency of general LLM knowledge for personalized privacy.

The evaluation consistently showed that Ariel provided a significant performance improvement across both appropriate and inappropriate F1 scores compared to all ICL and zero-shot baselines. This robust testing environment, using real-world user data and established LLM models, served as the proof of concept for Ariel's efficacy in personalizing agent privacy decisions via logical entailment.

Defensive Implications

▶ Watch: LLMs struggle to abstain when uncertain about privacy decisions (9:20)

The Ariel framework offers profound defensive implications for users, developers, and organizations building or deploying AI agents that handle sensitive personal data. Its core contribution is to provide a structured, interpretable, and user-aligned method for automating privacy decisions, addressing critical security and privacy challenges inherent in the rise of autonomous agents.

For AI agent developers, Ariel provides a robust blueprint for designing privacy-preserving systems. Instead of relying on ad-hoc rules or opaque LLM outputs, developers can integrate Ariel's logical entailment engine. This means:

  • Structured Privacy Logic: Developers can move beyond simple consent pop-ups to a system that intelligently infers user preferences based on historical decisions. This reduces the burden on users while maintaining alignment with their privacy posture.
  • Interpretability by Design: The use of ontologies and a clear entailment logic block means that the agent's privacy decisions are not black-box outputs. Developers can build interfaces that explain why a decision was made (e.g., "I shared your partial SSN because you previously approved sharing your full SSN for the same purpose and recipient, and partial SSN is less sensitive"). This transparency is crucial for user trust and debugging.
  • Reduced Risk of Over-sharing: By accurately identifying challenging or ambiguous requests and escalating them to the user, Ariel minimizes the risk of the agent making an inappropriate or unapproved data sharing decision. This is a critical defensive measure against potential privacy breaches stemming from agent autonomy.
  • Scalability for Diverse Data: Unlike static rule-based systems, Ariel's LLM-driven ontology generation, conditioned on user decisions, allows it to adapt to new data types and recipients without constant manual reprogramming. This makes it a more scalable solution for the rapidly evolving landscape of AI agent tasks.

For users, Ariel directly combats privacy fatigue while empowering them with greater control:

  • Fewer Interruptions: Users will experience significantly fewer prompts for routine or easily inferable privacy decisions, making their interaction with AI agents smoother and more efficient.
  • Enhanced Trust: The ability to review the agent's reasoning for a decision fosters trust. Users can understand the logic, correct the agent if it misinterprets their preferences, and feel confident that sensitive decisions are either handled appropriately or escalated.
  • Personalized Privacy: Ariel ensures that privacy decisions are tailored to the individual user's specific preferences, rather than relying on generic privacy norms. This truly personalizes the privacy experience, aligning the agent's behavior with the user's unique comfort levels.

For organizations deploying AI agents, Ariel offers a pathway to build more compliant and ethically sound systems:

  • Compliance Support: By demonstrating a clear, auditable logic for privacy decisions, Ariel can help organizations meet regulatory requirements for data privacy and transparency.
  • Ethical AI Development: Implementing Ariel aligns with principles of ethical AI, particularly those concerning user autonomy, transparency, and accountability in data handling.
  • Reputation Management: Proactively addressing privacy concerns through intelligent, user-centric mechanisms can enhance an organization's reputation and build user loyalty.

In essence, Ariel provides a robust defensive framework that transforms AI agents from potential privacy liabilities into trustworthy data stewards. It empowers agents to make intelligent, context-aware privacy decisions, reduces user fatigue, and ensures that the user remains the ultimate arbiter of their personal data, especially in ambiguous situations.

Key Takeaways

  • Logical Entailment for Personalized Privacy: Ariel introduces a novel framework that redefines privacy decision-making as a logical entailment problem, allowing AI agents to infer appropriate data sharing actions based on a user's prior preferences and contextual information.
  • Superior Performance over Pure LLMs: The system significantly outperforms purely LLM-based approaches (such as In-Context Learning and zero-shot models) in making accurate and personalized privacy judgments, demonstrating the limitations of relying solely on LLMs' general privacy norm awareness.
  • Effective Escalation for Ambiguous Cases: A critical strength of Ariel is its ability to accurately identify and escalate challenging or ambiguous data sharing requests to the user, ensuring that complex decisions are always handled by a human and preventing erroneous automated actions.
  • Balancing Automation and User Control: Ariel successfully addresses privacy fatigue by automating routine privacy decisions through structured reasoning, while simultaneously preserving user agency by escalating non-inferable or critical decisions, thereby maintaining user control over sensitive data.
  • Interpretable Decision-Making: By leveraging ontologies and a clear entailment logic block, Ariel provides interpretable reasoning for its privacy decisions, fostering user trust and allowing for transparency and auditing of agent behavior.

About the Speaker(s)

The research presented in this talk, "Personalizing Agent Privacy Decisions via Logical Entailment," is a collaborative effort by several distinguished academics and industry researchers.

James Flemings is listed as a paper author from the University of Southern California, and he was the presenter of this work at the PETS conference.

The paper's co-authors include:

  • Ren Yi from Google
  • Octavian Suciu from Google
  • Kassem Fawaz from the University of Wisconsin--Madison
  • Murali Annavaram from the University of Southern California
  • Marco Gruteser from Google

This diverse authorship, spanning both prominent academic institutions and a leading technology company, underscores the interdisciplinary nature of the research, combining expertise in privacy, AI, and systems design.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

We are handing autonomous LLMs the keys to user data, and relying on native model guardrails is a recipe for a catastrophic privacy breach. This research actually proposes a workable defensive architecture: using LLMs to build user-specific ontologies, but forcing the actual data-sharing decision through a verifiable, deterministic logic block.

Heather Calloway (CISO) — SOLID

This talk tackles a fundamental blocker to enterprise AI agent adoption: how to manage data privacy consent without drowning users in prompts or relying on unexplainable black-box models. It introduces a hybrid architecture that enforces deterministic logic over LLM assumptions, making privacy decisions auditable. It's required reading for your privacy engineering and AI governance leads.

→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026

All talks from Privacy Enhancing Technologies Symposium (PETS) 2026