Personal Data Flows and Privacy Policy Traceability in Third-party GPT Integrations
Juan Carlos Carrillo (Paper author · VRAIN, Universitat Politècnica de València), Jose Luis Martin Navarro (Paper author · Aalto University & VRAIN, Universitat Politècnica de València), Rongjun Ma (Paper author · Aalto University), Jose Such (Paper author · King’s College London & VRAIN, Universitat Politècnica de València)
Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 3 · Caribou
Overview
In an era defined by the rapid proliferation of artificial intelligence, the emergence of custom GPTs has democratized the creation of specialized AI applications, allowing anyone to build tailored ChatGPT versions without coding expertise. However, this accessibility introduces significant, often overlooked, privacy risks, particularly when these custom GPTs integrate with external third-party services. This talk, presented by Juan Carlos Carrillo at PETS, delves into the critical issue of personal data flows from these custom GPTs to third-party APIs and rigorously assesses the traceability and accuracy of their associated privacy policies.

Key moments
- 0:40 Understanding custom GPTs and external integrations
- 2:10 Research questions on data transmission and disclosure
- 4:00 Creating a personal data taxonomy for LLM apps
- 5:00 Key finding: 35% of API parameters expose personal data
- 6:20 Analyzing privacy policies for data traceability
- 7:00 Key finding: 85% of GPTs fail full data disclosure
- 8:10 Recommendations for platforms, users, and creators
Personal Data Flows and Privacy Policy Traceability in Third-party GPT Integrations
Speakers: Juan Carlos Carrillo (Paper author, VRAIN, Universitat Politècnica de València); Jose Luis Martin Navarro (Paper author, Aalto University & VRAIN, Universitat Politècnica de València); Rongjun Ma (Paper author, Aalto University); Jose Such (Paper author, King’s College London & VRAIN, Universitat Politècnica de València)
Conference: PETS
YouTube: https://www.youtube.com/watch?v=f8OorYOP9ZA
Overview
In an era defined by the rapid proliferation of artificial intelligence, the emergence of custom GPTs has democratized the creation of specialized AI applications, allowing anyone to build tailored ChatGPT versions without coding expertise. However, this accessibility introduces significant, often overlooked, privacy risks, particularly when these custom GPTs integrate with external third-party services. This talk, presented by Juan Carlos Carrillo at PETS, delves into the critical issue of personal data flows from these custom GPTs to third-party APIs and rigorously assesses the traceability and accuracy of their associated privacy policies.
The research presented highlights a pervasive failure in how personal data is handled and disclosed within the burgeoning GPT ecosystem. With millions of custom GPTs now in circulation, many of which are designed to interact with external services, understanding the true extent of data transmission and the transparency of privacy disclosures is paramount. This work moves beyond traditional analyses of LLM vulnerabilities to expose a fundamental disconnect between user expectations, developer practices, and platform requirements concerning personal data protection.
The findings reveal that a substantial portion of custom GPTs transmit personal data to third parties, often without adequate or accurate disclosure in their privacy policies. This exposes users to unforeseen privacy risks, as their sensitive information may be shared with external entities without their informed consent. The research not only quantifies the scale of this problem but also provides actionable insights and recommendations for platforms, creators, and users to foster a more transparent and privacy-respecting AI landscape.
Background
▶ Watch: Understanding custom GPTs and external integrations (0:40)
The landscape of conversational AI has been significantly transformed by the advent of Large Language Models (LLMs) and, more recently, custom versions like OpenAI's Custom GPTs. These custom GPTs enable users to create specialized instances of ChatGPT for particular purposes, often without requiring any programming knowledge. This ease of creation has led to an explosion in their numbers, with over 3 million GPTs created by January 2024, attracting approximately 6.1 million visits per month. Examples range from seemingly innocuous tools like resume builders to more sensitive applications like "find my celebrity look-alike," which might involve uploading personal photos.
The architecture of a custom GPT, while seemingly straightforward, harbors potential privacy pitfalls. Beyond a system prompt that defines its behavior and contextual files, the most critical aspect for data privacy is its ability to connect to external, third-party services via Actions. Developers configure these actions by providing a JSON schema, which essentially serves as a blueprint. This schema defines the API endpoints the model should call, the specific parameters it should retrieve from the user's conversation, and the data types associated with those parameters. Crucially, the model then interprets when and how to call these external APIs based on user input and the defined schema.
OpenAI, as the platform provider, mandates that custom GPT creators provide a privacy policy that outlines how user data will be managed. However, the efficacy and adherence to this requirement have remained largely unexamined. Existing literature concerning GPTs primarily focuses on direct vulnerabilities (e.g., prompt injection) or the creation of malicious GPTs. There has been a significant gap in understanding the specific problem of personal data transmission from these custom GPTs to third-party services and whether their privacy policies accurately reflect these data practices. This research directly addresses this gap, focusing on two core questions: what personal information is being transmitted through custom GPTs to external parties, and if their privacy policies actually disclose what is sent.
Key Findings
▶ Watch: Creating a personal data taxonomy for LLM apps (4:00)
The research yielded several critical findings that underscore the pervasive privacy challenges within the custom GPT ecosystem. These findings highlight significant discrepancies between declared privacy practices and actual data handling, posing substantial risks to user privacy.
Firstly, the study revealed that a considerable proportion of API parameters configured within custom GPTs are designed to transmit personal data. Specifically, 35% of all API parameters exposed personal data. This figure is alarming, indicating that a substantial amount of sensitive user information, ranging from names and dates to potentially more intimate details depending on the GPT's purpose, is being channeled to external third-party services. This transmission often occurs without users being fully aware of the extent or nature of the data shared.
Secondly, the analysis of privacy policies demonstrated a widespread failure in traceability and disclosure. The researchers categorized privacy policy adherence into three levels:
- Complete Traceability: Only 15% of the audited GPTs had privacy policies that fully disclosed all the personal data being collected and transmitted to third parties. This low percentage indicates that comprehensive transparency is a rare exception rather than the norm.
- Partial Traceability: A significant 42.7% of GPTs provided only partial disclosure. This category includes policies that might mention collecting "personal information" in a vague manner without specifying the exact data types or purposes, or policies that disclose some but not all of the collected data. Such ambiguity leaves users in the dark about specific data practices.
- Broken Policies: An equally concerning 42.3% of GPTs were found to have entirely broken policies. This means either no privacy policy was provided, or the link to the policy was non-functional, rendering any attempt at understanding data practices impossible. This high percentage of non-existent or inaccessible policies represents a severe breach of trust and a fundamental failure to meet basic privacy requirements.
The researchers also identified potential reasons behind these widespread failures. One contributing factor is the assumption by many custom GPT creators that OpenAI automatically sanitizes user data before it is sent to third parties. This assumption, as demonstrated by the study's active audit, is often incorrect. Another significant reason is the over-reliance on generic policy generators. These tools often produce boilerplate text that fails to accurately reflect the specific and dynamic data collection practices configured via a GPT's JSON schema, leading to a disconnect between policy and practice. These findings collectively paint a concerning picture of a rapidly expanding AI ecosystem where privacy protections are critically underdeveloped and often neglected.
Technical Deep Dive
▶ Watch: Key finding: 35% of API parameters expose personal data (5:00)
The methodology employed in this research provides a robust framework for analyzing personal data flows and privacy policy traceability in the custom GPT ecosystem. The process was meticulously structured, beginning with extensive data collection, followed by the development of a novel personal information taxonomy, and culminating in sophisticated classification and traceability analyses.
The data collection phase was comprehensive. The researchers initially crawled over 340,000 GPTs available in the ecosystem. From this vast dataset, they filtered down to custom GPTs that had Actions enabled, indicating their ability to send data to external third-party services. This crucial filtering step resulted in a subset of 5,286 GPTs that were capable of external data transmission, forming the core of their study. For each of these GPTs, the associated JSON schema was extracted. The JSON schema is pivotal as it explicitly defines the structure and content of the data sent to external APIs, including the target endpoint, the data type of each parameter, and a data description that helps the LLM interpret what information to extract from the conversation.
A significant challenge identified was the absence of a personal information taxonomy specifically adapted to the nuanced context of conversational AI. To address this, the researchers developed their own taxonomy, grounding it in established privacy guidelines such as NIST (National Institute of Standards and Technology) and GDPR (General Data Protection Regulation). This rigorous approach led to the creation of 83 distinct labels for classifying various types of personal information, providing a granular framework for subsequent analysis.
For the parameter classification methodology, the researchers undertook a substantial annotation effort, manually labeling over 700 cases of API parameters using their newly developed taxonomy. This annotated dataset was then used to train and evaluate machine learning models. Intriguingly, initial attempts with large language models, specifically GPT-4, yielded a macro F1 score of only 61%. This highlights a potential limitation of off-the-shelf LLMs for highly specific and precise classification tasks without extensive fine-tuning. In contrast, fine-tuning a smaller language model, Roberta, with their custom annotated dataset achieved a significantly higher macro F1 score of 87%. This finding is a crucial technical insight, demonstrating that specialized, fine-tuned models can outperform larger, general-purpose LLMs for domain-specific classification, particularly when dealing with structured data descriptions within JSON schemas.
The traceability analysis focused on assessing the accuracy of privacy policies. For each unique GPT, its associated privacy policy was collected. To systematically analyze these policies, the researchers employed Polygraph, a natural language processing (NLP) tool designed to extract collection statements from privacy policies. These extracted statements, detailing what data the policy claimed to collect, were then mapped to their personal information taxonomy using an LLM. This mapping allowed for a direct comparison between the data declared in the policy and the data actually configured for transmission via the JSON schemas. Policies were then classified into three categories: Complete (all collected data disclosed), Partial (some data disclosed or vague language used), and Broken (no policy or non-functional link). This multi-stage technical approach ensured a comprehensive and verifiable assessment of data flows and privacy disclosures.
Demo / Proof of Concept
▶ Watch: Key finding: 85% of GPTs fail full data disclosure (7:00)
While not a live demonstration in the traditional sense, the talk included a compelling "active audit example" that served as a proof of concept, vividly illustrating how personal data is transmitted from custom GPTs to external parties. This example directly validated the findings regarding personal data exposure via API parameters.
The researchers described replicating their own custom GPTs to meticulously observe the data flow. In this setup, a user would input text into the GPT's prompt. The critical element was the JSON schema defined for the GPT's external action. Within this schema, an API parameter was explicitly named original text. The description field for this parameter was particularly revealing, stating: "Please provide the original request only containing user input that triggered the API call."
Furthermore, the developer configuring this GPT had explicitly noted that this parameter could contain "sensitive user data such as name, date." This developer annotation within the JSON schema itself underscored the potential for personal data transmission. The crucial step in the proof of concept was the verification that when a user indeed provided input containing sensitive information (e.g., their name and a date) into the GPT, the original text parameter, as defined by the JSON schema, actually sent this exact user input to the external third-party server.
This active audit served as a definitive confirmation of the first research question: personal data, as specified and intended by the GPT's developer through the JSON schema, is indeed transmitted to external parties. It highlighted that the JSON schema acts as a direct conduit for user data, and if developers configure it to extract sensitive information, that data will be sent, irrespective of any assumptions about platform-level sanitization. This concrete example provided tangible evidence of the privacy risks inherent in poorly configured or undisclosed third-party GPT integrations.
Defensive Implications
▶ Watch: Recommendations for platforms, users, and creators (8:10)
The findings of this research carry significant defensive implications for all stakeholders involved in the custom GPT ecosystem: platforms, users, and creators. Addressing these vulnerabilities requires a multi-pronged approach to enhance transparency, accountability, and user protection.
For Platforms (e.g., OpenAI):
The primary platform provider plays a crucial role in shaping the security and privacy posture of the ecosystem. The research strongly recommends implementing strict validation checks for privacy policies during the GPT creation process. This would involve more than just a simple link check; platforms should consider automated tools that can analyze the content of the provided privacy policy and cross-reference it with the configured JSON schemas for external actions. Such tools could identify discrepancies where the policy fails to disclose all data types specified for transmission. Furthermore, platforms could enforce clearer guidelines or even automated warnings if a JSON schema is configured to send personal data without explicit, matching disclosures in the privacy policy. This would shift the responsibility from mere declaration to verifiable adherence.
For Users:
Users are currently at a significant disadvantage due to the lack of transparency. A key recommendation is the implementation of pre-execution data previews. Before a custom GPT initiates an external API call that involves transmitting user data, the platform should display a clear, concise summary of what specific data (e.g., "your name," "your location," "your uploaded photo") is about to be sent to which external service. This would empower users with informed consent, allowing them to approve or deny the data transmission on a case-by-case basis. Additionally, broader educational campaigns could raise user awareness about the privacy implications of interacting with third-party GPTs and encourage them to scrutinize privacy policies more closely, or to be cautious about sharing sensitive information.
For Creators:
Developers and creators of custom GPTs bear direct responsibility for the data handling practices of their applications. The research highlights an urgent need for educational resources on data handling responsibilities. Many creators may not fully understand the implications of their JSON schema configurations, or they might mistakenly assume that platforms automatically sanitize user data. Comprehensive guides, best practices, and even mandatory training modules could help creators understand data privacy principles (e.g., GDPR, NIST guidelines) and how to accurately reflect their data practices in clear, specific, and complete privacy policies. Discouraging the sole reliance on generic policy generators and instead promoting the creation of tailored, accurate policies is also crucial. Creators must be made aware that they are accountable for the data their GPTs transmit and that transparency is not merely a formality but a fundamental ethical and legal obligation.
By implementing these defensive measures, the custom GPT ecosystem can evolve towards a more secure and privacy-respecting environment, fostering trust and enabling users to leverage the power of specialized AI tools with greater confidence.
Key Takeaways
- A substantial portion of custom GPTs (35% of API parameters) are configured to transmit personal data to external third-party services, highlighting a significant privacy exposure risk.
- The vast majority (85%) of custom GPTs exhibit critical failures in privacy policy transparency, with only 15% achieving complete disclosure of their data collection and transmission practices.
- A concerning 42.3% of custom GPTs operate with entirely broken, missing, or inaccessible privacy policies, rendering user understanding of data handling impossible.
- Many custom GPT creators mistakenly assume that platform providers automatically sanitize user data, or they over-rely on generic privacy policy generators that fail to accurately reflect specific data flows.
- Platforms like OpenAI must implement stricter validation checks for privacy policies and consider pre-execution data previews to inform users about data transmission.
- There is an urgent need for comprehensive educational resources for custom GPT creators to ensure they understand their data handling responsibilities and the importance of accurate, transparent privacy policies.
About the Speaker(s)
The research presented in this talk was a collaborative effort by a team of academics. The presentation at PETS was delivered by Juan Carlos Carrillo, who is identified as a paper author affiliated with VRAIN, Universitat Politècnica de València. This marked his first time presenting at the PETS conference. His co-authors include Jose Luis Martin Navarro, also a paper author associated with Aalto University and VRAIN, Universitat Politècnica de València. Rongjun Ma is another paper author from Aalto University. Finally, Jose Such, serving as a supervisor for the project, is a paper author affiliated with King’s College London and VRAIN, Universitat Politècnica de València. The team's collective expertise spans privacy, AI, and security research, contributing to this critical analysis of the custom GPT ecosystem.
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
Carrillo and team audited the Custom GPT ecosystem and proved what we all suspected: it's a massive, unregulated PII vacuum. By scraping JSON schemas from 5,000 GPTs with external actions, they showed 85% have garbage privacy policies while quietly funneling user data to third-party APIs. It's a rigorous, empirical reality check on OpenAI's marketplace governance.
Heather Calloway (CISO) — MUST SEE
This research quantifies a massive compliance blind spot in enterprise AI adoption: custom GPTs routinely siphon user data to external APIs under broken or non-existent privacy policies. It provides the exact empirical backing a CISO needs to justify locking down third-party GPT actions across the corporate footprint.
→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026
All talks from Privacy Enhancing Technologies Symposium (PETS) 2026