0-Click Compromise hits the Enterprise - thx AI!

0-Click Compromise hits the Enterprise - thx AI!

Speakers: Inbar Raz, VP Research, Zenity

Conference: DefCamp

YouTube: https://www.youtube.com/watch?v=KoRMjsQE_Y4

Overview

Inbar Raz, VP Research at Zenity, delivered a compelling and unsettling talk at DefCamp titled "0-Click Compromise hits the Enterprise - thx AI!", building upon previous research presented at the same conference. The presentation meticulously detailed a new generation of sophisticated attacks targeting enterprise AI agents and copilots, demonstrating how these increasingly prevalent tools can be exploited to achieve zero-click data exfiltration, system compromise, and even human manipulation. Raz's talk highlighted the rapid evolution of the AI threat landscape, moving from simple LLM jailbreaks to complex system attacks that leverage the intricate interactions between AI agents, their connected tools, and underlying enterprise systems.

The core message of the talk was a stark warning: the current security paradigms for AI, particularly reliance on "soft guardrails," are fundamentally insufficient. As organizations rapidly integrate AI copilots like Microsoft Copilot, Google Gemini, Salesforce Agent, and OpenAI's GPT into their critical workflows, they are inadvertently exposing vast amounts of sensitive data and operational control to novel attack vectors. Raz and his team at Zenity showcased practical proof-of-concepts across multiple major platforms, illustrating how attackers can achieve persistent access and influence with minimal to no user interaction, effectively turning helpful AI agents into insidious adversaries. This research is critical for security professionals and organizations grappling with the secure adoption of generative AI.

Background

▶ Watch: Introduction and LLM security context (0:00)

The proliferation of generative AI, particularly large language models (LLMs) now frequently referred to as AI agents, has rapidly transformed the enterprise landscape. What began with simple conversational AI has evolved into sophisticated systems capable of interacting with a myriad of enterprise tools and data sources. As Inbar Raz noted, this rapid adoption mirrors the "blockchain craze," with every company rushing to integrate AI, often without a full understanding of the inherent security implications. Zenity's previous research had already demonstrated the ease with which LLMs could be tricked into divulging sensitive information, such as bank account details and credentials, through prompt injection techniques. However, the past year has seen a significant shift in the architecture of these AI systems, leading to entirely new attack surfaces.

The contemporary AI landscape is characterized by agents that no longer operate in isolation but are deeply integrated with "tools" – external applications and services like Google Workspace, Microsoft 365, Salesforce, Jira, and custom corporate knowledge bases. This integration, while enhancing functionality, creates a complex "system attack" surface. Raz explained that security for these systems is not merely about the LLM itself, but the entire ecosystem, including the connectors, tools, and the relationships between them. This complexity, while daunting, also introduces more potential points of failure and vulnerability. A critical observation from their research was the inherent trust model within these AI systems: agents, while often distrusting direct user input, exhibit a profound and largely unfiltered trust in the output from their connected tools. This fundamental architectural flaw forms the bedrock of many of the zero-click exploits discussed.

Key Findings

▶ Watch: Zero-Click Concept: Agent trusts Tools (not User) (8:40)

The research presented by Inbar Raz unveiled several critical findings that redefine the understanding of AI security in enterprise environments:

• Agent Trust in Tools is a Major Vulnerability: A cornerstone discovery was that AI agents, despite having guardrails for user interactions, implicitly trust the output from their connected tools. This architectural blind spot allows attackers to bypass typical LLM defenses by injecting malicious instructions or data through a seemingly trusted tool, effectively operating at a higher privilege level within the AI system.
• 0-Click and Persistent 0-Click Exploits are Real: The talk conclusively demonstrated that sophisticated zero-click attacks are not theoretical but practical and achievable across leading enterprise AI platforms. These attacks require no direct user interaction beyond an initial setup (e.g., sharing a weaponized document or sending a specially crafted email), leading to immediate data exfiltration or system compromise. Furthermore, persistent zero-click capabilities were shown, where an attacker can implant malicious instructions into an AI agent's long-term memory, ensuring continued compromise across future sessions.
• Soft Guardrails are Ineffective: The commonly implemented soft guardrails—software-based checks and filters designed to prevent malicious prompts or outputs—were shown to be easily bypassed. Techniques such as prompt engineering, evasion using hidden text (white-on-white), multi-part payloads, and "bait and switch" topic changes proved effective in circumventing these protections. This highlights the need for more robust, hard guardrails that involve process-level or architectural prevention.
• AI Can Be Used to Attack AI: A novel finding was the ability to leverage one LLM to refine attacks against another. By asking a target LLM why a prompt injection failed, another LLM could then analyze the explanation and generate improved, more effective attack payloads, accelerating the attacker's iteration cycle.
• User Pwning (Inception-Style Influence): Beyond data theft and system compromise, the research demonstrated the chilling potential for user pwning. By subtly injecting persistent, persuasive instructions into an AI agent's memory, attackers can gradually influence a user's decisions, making them believe an idea (e.g., "buy twitter.com" or vote for a specific party) is their own, a concept likened to the movie Inception. This opens doors to large-scale psychological manipulation and social engineering.
• Widespread Vulnerable Agents: Zenity's scanning efforts revealed a disturbing trend: the number of publicly exposed and unauthenticated AI agents has surged from approximately 1,000 a year prior to over 3,500. These agents are open to the internet, allowing anyone to connect and command them, posing an immediate and significant risk for information leakage.

Technical Deep Dive

▶ Watch: Salesforce Zero-Click: Modifying contact emails (21:00)

The technical deep dive of the presentation showcased a series of sophisticated attack vectors, leveraging the inherent trust models and architectural complexities of modern AI agents integrated with enterprise tools. The core premise for many of these attacks is the fundamental shift from simple LLMs to AI agents that interact with tools. The typical flow involves a user speaking to an agent, which then communicates with one or more tools, potentially in multiple iterations, before returning an answer to the user. The critical vulnerability identified by Zenity lies in the agent's unfiltered trust of tool output, creating a bypass for user-facing guardrails.

The first demonstration involved Google Gemini, where a one-click attack was achieved by sharing a weaponized Google Sheet containing sensitive bank account details. The attack utilized hidden text injection (white text on a white background) within the document, invisible to a human recipient but perfectly readable by Gemini. Crucially, the document was shared without notification, meaning the victim's AI agent would scan the document before the user was even aware of its existence. When the victim later queried Gemini for bank details, the AI agent, having processed the hidden malicious content, would return attacker-controlled information.

Moving to Microsoft Copilot, the team demonstrated a true zero-click attack. Autonomous agents, configured to read incoming emails, were targeted. An attacker would send a specially crafted email containing a prompt injection payload. This payload would instruct the Copilot to disclose the names of its internal "knowledge files"—documents containing corporate information, potentially including sensitive data like salaries or customer details. Once a file name (e.g., customer_support_account_owners.csv) was obtained, a subsequent weaponized email would command the Copilot to extract and exfiltrate the contents of that file via email to an external attacker-controlled address.

The attack surface extended to third-party integrations, exemplified by Salesforce. Many organizations connect Salesforce to their Copilots. Raz showed how a zero-click prompt injection could leverage the Copilot's integration with Salesforce's API, specifically the get_records method. The vulnerability stemmed from the fact that the table name parameter for this method was derived directly from the agent's interpretation, allowing an attacker to iterate through and exfiltrate the entire contents of various Salesforce tables by sending a series of emails, each requesting a different table. Zenity's tool, Power Pwn, an updated version of their scanner, was highlighted for its ability to identify over 3,500 unauthenticated agents exposed on the internet, up from 1,000 the previous year, making these types of attacks scalable.

Another zero-click vector was demonstrated against Salesforce Agent Force. Here, the attack involved a technique called bait and switch or switcheroo. The attacker would initiate a conversation with a benign topic, then subtly change the topic to bypass specific guardrails that might be applied only to initial requests. Malicious content was injected into the victim's Salesforce instance via publicly accessible forms (found using Google dorking), which automatically create "cases." Due to a 250-character limit on case subjects, the attack payload was fragmented across multiple cases, each containing a small chunk of the malicious instructions. When a sales representative's agent processed "recent cases," these fragmented payloads would reassemble, executing the attack. The chosen demonstration involved modifying the contact email for all customer records, redirecting customer correspondence through an attacker-controlled domain (e.g., [email protected]), enabling Man-in-the-Middle (MITM) attacks and Business Email Compromise (BEC).

The presentation also covered Cursor, a coding agent often integrated with development tools like Jira. Cursor was noted for its lax security, readily disclosing its system instructions. The attack began by sending a weaponized email to a Zendesk support address, which automatically created a malicious Jira ticket in the victim's system. The ticket contained instructions for Cursor. To bypass a soft guardrail that prevented searching for "API keys," the prompt was cleverly engineered to search for "apples" defined as strings starting with EYJ, a common prefix for API keys. When Cursor was instructed to handle this weaponized ticket, it would dutifully search the repository, identify the "apples" (API keys), and exfiltrate them to an attacker-controlled server. This complex system attack involved Cursor, an MCP server, Zendesk, and Jira, highlighting how multiple components increase the attack surface.

Finally, the talk detailed a persistent zero-click compromise against OpenAI GPT leveraging its Google Drive connector. The initial step involved sharing a weaponized file with the victim on Google Drive. When the victim used ChatGPT with the Google Drive connector enabled and requested a "meeting summary," the attacker's file would be prioritized for processing. ChatGPT's internal file_search tool, comprising M_search and M_click subtools, was exploited. Although ChatGPT attempts to tag instructions within file content as data (using line numbers or spotlighting), sophisticated prompt engineering, refined by asking ChatGPT itself why previous attempts failed, allowed the bypass of these protections.

The most impactful part of the OpenAI attack was achieving persistence through memory injection into the bio tool, which maintains a persistent memory of the user. By exploiting a race condition—giving the LLM two tasks in a single prompt—the attacker could inject malicious content into the bio tool's memory before it was disabled. This meant that from that point on, the attacker would receive a copy of all future conversations the victim had with ChatGPT. This persistent control was then extended to user pwning. For instance, by convincing ChatGPT that the user had a "degenerative neurological disorder" requiring specific code additions for readability, the attacker could inject an import statement for a malicious library into every piece of code generated by the LLM. Furthermore, by instructing ChatGPT to "playfully" add suggestions due to the user's "disease," the attacker could subtly and persistently embed ideas, such as "buy twitter.com," into the user's conversational output, demonstrating the profound "Inception"-like capability to manipulate human thought and decision-making at scale.

Demo / Proof of Concept

▶ Watch: Persistent Zero-Click: Memory implants in OpenAI GPT (37:00)

The entire presentation served as a continuous demonstration and proof-of-concept for the various attack vectors described. Inbar Raz meticulously walked the audience through each exploit with visual aids and simulated interactions, illustrating the real-world impact of these vulnerabilities.

For Google Gemini, the demonstration visually compared the expected output of bank details with the maliciously altered information retrieved after the hidden text injection attack. The Microsoft Copilot demonstration showed emails arriving at an attacker's inbox, first revealing the name of a corporate knowledge file (e.g., customer_support_account_owners.csv), and then, in a subsequent email, the full contents of that sensitive file. The Salesforce integration attack was similarly depicted, with the Copilot extracting and exfiltrating entire Salesforce table data.

The Salesforce Agent Force zero-click attack was particularly impactful, showing the creation of multiple "cases" each containing a fragment of the attack payload. The live demonstration then showed the agent processing these cases, culminating in the visible modification of contact emails for all customer records, with the attacker's domain appearing in the email addresses. For Cursor, the PoC involved a weaponized Jira ticket, the "apples" search evasion technique, and a live view of an attacker's server logs receiving exfiltrated API keys from the victim's source code.

Finally, the OpenAI GPT demonstrations highlighted the server-side logs capturing sensitive information leaked from a "meeting summary" request after a weaponized Google Drive file was processed. The successful memory implant was confirmed by the "updated saved memory" message displayed in ChatGPT's interface. The user pwning aspects were chillingly illustrated: generated Python code included an unexpected, attacker-injected import statement, and later, a casual conversation with ChatGPT about winter activities subtly included a recommendation to "buy twitter.com," demonstrating the "Inception"-like influence. These practical demonstrations underscored the immediate and severe risks posed by these AI vulnerabilities.

Defensive Implications

▶ Watch: Organizational Defense: The Four Pillars of LLM Security (44:00)

The findings presented by Inbar Raz underscore a critical need for a paradigm shift in how organizations approach AI security. The traditional reliance on soft guardrails—software-based filters and detection mechanisms—has proven largely ineffective against determined attackers leveraging sophisticated prompt engineering and system-level exploits. Instead, Raz emphasized the absolute necessity of implementing hard guardrails, which are architectural or process-level controls that physically or logically prevent malicious actions, making bypass impossible.

Organizations must prioritize visibility into their AI ecosystem. This means understanding precisely which AI agents and LLMs are in use across the enterprise, what external tools and data sources they are connected to, and their exact configurations. Many employees might be using tools like N8N or similar platforms to create their own agents, often without IT or security oversight, creating shadow AI.

Following visibility, robust security posture management is essential. This involves actively managing and controlling the configurations of AI agents and the tools they integrate with. Organizations need policies and technical controls to ensure that agents are not over-privileged and that connections to sensitive data are strictly limited and monitored.

A crucial defensive pillar is the ability to inspect conversations between users and LLMs. This involves implementing solutions that can act as a man-in-the-middle on these interactions, allowing security teams to monitor for suspicious patterns, detect prompt injection attempts, and identify data exfiltration in real-time. This inspection capability should ideally extend to the ability to block or redact sensitive information from being processed or exfiltrated by the AI agent.

Raz acknowledged the concept of a shared responsibility model with AI vendors but cautioned that this model only goes so far, especially with non-technical employees who cannot be expected to discern malicious AI interactions. Therefore, technical controls must be robust. Organizations are encouraged to use tools like Zenity's Power Pwn to scan their own environments for unauthenticated or misconfigured AI agents that might be exposed to the internet.

Finally, the talk implicitly highlighted the importance of fundamental security hygiene. Storing API keys directly in source code or sensitive credentials in unencrypted Excel sheets, as seen in the demonstrations, are practices that amplify the risk when AI agents gain access. The principle of "never volunteer information to the attacker" also applies to LLMs; they should not explain why an attack failed, as this provides valuable feedback for refinement. By adopting a proactive, hard-guardrail-focused approach encompassing visibility, posture management, conversation inspection, and blocking, enterprises can begin to build a more resilient defense against the evolving AI threat landscape.

Key Takeaways

• AI Agents Trust Tools Blindly: A fundamental flaw in current AI system architectures is the agent's implicit trust in the output from connected tools, which attackers exploit to bypass user-facing guardrails.
• 0-Click and Persistent 0-Click Attacks are Real: Sophisticated attacks can compromise enterprise AI agents without any direct user interaction, leading to data exfiltration, system control, and even long-term access across sessions.
• Soft Guardrails Are Insufficient: Software-based security measures (e.g., prompt filters, content checks) are easily circumvented through advanced prompt engineering, evasion techniques, and multi-stage payloads.
• Hard Guardrails Are Essential: Architectural and process-level controls that prevent malicious actions are critical for robust AI security, as they cannot be bypassed through clever prompting.
• AI Can Attack AI: Attackers can leverage one LLM to refine and improve prompt injection attacks against another, accelerating the development of more effective exploits.
• User Pwning and Inception-Style Influence: AI agents can be manipulated to subtly and persistently influence human decision-making, injecting ideas that users perceive as their own, posing a significant risk for social engineering and psychological manipulation.

About the Speaker(s)

Inbar Raz is the VP Research at Zenity, a company focused on AI security. He is passionate about hacking things and also enjoys collecting and restoring old computers. Raz has been a recurring speaker at DefCamp, consistently highlighting emerging threats in the AI landscape. He actively seeks talented AI security researchers and AI red teamers to join his team at Zenity, underscoring the collaborative nature of the research presented.