Malicious GenAI Chrome extensions: unpacking data exfiltration and malicious behaviours

Malicious GenAI Chrome extensions: unpacking data exfiltration and malicious behaviours

Speakers: Shresta B. Seetharam (Palo Alto Networks); Mohamed Nabeel (Palo Alto Networks); William Melicher (Palo Alto Networks)

Conference: Virus Bulletin

YouTube: https://www.youtube.com/watch?v=6duDjiWEJOw

Overview

This talk, presented by Shresta B. Seetharam from Palo Alto Networks, alongside co-authors Mohamed Nabeel and William Melicher, delves into the burgeoning threat landscape of malicious Generative AI (GenAI) Chrome extensions. The presentation unpacks how attackers are exploiting the surging demand for AI-powered browser functionalities to conduct sophisticated data exfiltration and malicious redirection campaigns. Seetharam highlights the disruptive impact of GenAI features on the extension ecosystem, revealing a diverse array of attacker tactics ranging from subtle data theft to overt affiliate fraud.

The research presented is critical for understanding the evolving web security challenges posed by the rapid adoption of AI. As users increasingly seek to enhance productivity with AI tools, the browser extension market has become a fertile ground for threat actors. Palo Alto Networks' findings, based on an analysis of over 150 malicious extensions, underscore the urgent need for heightened vigilance from users, developers, and security vendors alike to counter these novel and often stealthy threats. The talk provides a comprehensive technical breakdown of these attacks, offering actionable insights into their mechanisms and defensive implications.

Background

▶ Watch: Introduction: Malicious GenAI Chrome extensions threats (0:00)

The advent of Generative AI has profoundly reshaped the digital landscape, with its influence rapidly extending into browser extensions. Users are exhibiting an insatiable demand for AI-driven features, seeking to integrate functionalities like summarization, email drafting, and enhanced search directly into their browsing experience. This demand is not merely anecdotal; market reports project the AI browser market to expand at a 33% Compound Annual Growth Rate (CAGR) over the next decade. Complementing this, Google SEO data indicates a staggering 900% year-over-year increase in searches for keywords like "Google extension AI," reflecting a widespread user desire for such tools. This explosive demand has led to a corresponding supply, with hundreds of new AI-powered extensions launching weekly on platforms like the Chrome Web Store.

However, this rapid proliferation has created a significant attack surface. To fully grasp the nature of these threats, it's essential to understand the fundamental architecture of browser extensions. Chrome extensions are essentially software add-ons built using standard web technologies: HTML, CSS, and JavaScript. They comprise five key components:

1. Service Worker (Background Script): This runs persistently in the background, handling core extension logic and events. In older Manifest V2, it was known as the background script.
2. Content Script: Injected directly into web pages, this script has privileged access to the Document Object Model (DOM), allowing it to read, write, and modify page content.
3. Pop-up: A small UI window that appears when the extension icon is clicked, often used for configuration or displaying quick information.
4. Manifest.json: A mandatory configuration file that defines the extension's metadata, permissions, scripts, and overall behavior. This file is crucial for security analysis.
5. CRX File Format: The compressed package format used for distributing extensions.

A critical architectural feature abused by attackers is message passing between the content script and the background script. While the content script can read sensitive data from the DOM, it typically lacks direct network access to external servers. Conversely, the background script can make network requests but cannot directly access the DOM of web pages. Message passing bridges this gap, enabling the content script to extract data from a webpage and forward it to the background script, which can then exfiltrate it to an attacker-controlled server.

The Manifest.json file also plays a pivotal role in security, as it declares the permissions an extension requests. Broad permissions, such as "all URLs" for host permissions (allowing communication with any internet resource) or injecting content scripts into "all URLs," significantly expand an extension's capabilities and, by extension, its potential for malicious abuse. Similarly, the ability to "execute JavaScript from any domain" grants immense power. While users are typically presented with a summary of requested permissions during installation, the sheer volume and technical nature of these permissions often lead to users granting overly broad access without full comprehension, creating an environment ripe for exploitation by malicious GenAI extensions.

Key Findings

▶ Watch: Key components and architecture of browser extensions (6:20)

Palo Alto Networks' research uncovered a significant campaign involving over 150 malicious GenAI Chrome extensions that exploit the demand for AI features. These extensions were found to fall into two broad categories of malicious behavior: data exfiltration and malicious redirection.

Within the realm of data exfiltration, the study identified several sophisticated attacker tactics:

• Adversary in the Browser (AiTB): Extensions directly monitor and steal sensitive user data from web pages, including confidential emails, messages, and even one-time passcodes (OTPs), often without explicit user interaction.
• Impersonation: Malicious extensions mimic legitimate, popular AI services or brands, leveraging familiar logos and user interfaces to trick users into installation.
• Dual Functionality: To evade detection and maintain user trust, these extensions often preserve their advertised, benign functionality while simultaneously performing malicious actions in the background.
• Bait and Switch: Attackers initially release benign versions of extensions, only to update them later with malicious capabilities, often involving re-obfuscation to hide the changes.

For malicious redirection, the research highlighted:

• Prompt Hijacking: User search queries or AI prompts are intercepted and redirected, often to attacker-controlled domains before reaching legitimate services, enabling data collection and potential manipulation.
• Affiliate Fraud: Extensions abuse browser APIs to automatically redirect users to affiliate links on installation, generating illicit revenue for attackers.
• PUP (Potentially Unwanted Program) Delivery: Extensions serve as conduits to drive traffic to grayware sites that trick users into installing unwanted software, adware, or further malicious extensions.

These findings collectively paint a picture of a rapidly evolving threat landscape where attackers are leveraging user trust in AI and exploiting the architectural nuances of browser extensions for financial gain and data theft, often with a low detection rate by traditional security vendors.

Technical Deep Dive

▶ Watch: Impersonation, dual functionality, and bait-and-switch (17:00)

The technical depth of these attacks lies in their sophisticated abuse of browser extension architecture and APIs, often employing stealthy methods to exfiltrate data or redirect users.

Data Exfiltration Techniques

1. Adversary in the Browser (AiTB) – The Supersonic AI Example:

This technique exemplifies direct data theft from the user's browser session. Consider the "Supersonic AI" extension, advertised as a productivity tool for email generation and summarization.

• Scenario: A user, "Frank," installs Supersonic AI. When he receives a confidential email (e.g., about a "Zebra Works" acquisition) and uses the extension to draft a reply, the malicious activity begins.
• Mechanism: The content script of Supersonic AI, injected into Frank's email client page, has access to the DOM. When Frank clicks "generate email," the content script captures the entire email content, including the confidential details, into a variable like last email content. This sensitive data is then passed via message passing to the background script.
• Exfiltration: The background script, which has network access, constructs an XML fetch or HTTP request containing the stolen email content and sends it to an attacker-controlled endpoint (e.g., API.Supersonic.email). Network logs in a sandbox environment clearly demonstrate this exfiltration.
• Passive Exfiltration: Even more insidiously, the extension can exfiltrate data without explicit user interaction. If Frank receives an email containing a One-Time Passcode (OTP) from LinkedIn, the "email summary" feature of Supersonic AI can automatically capture and exfiltrate this OTP to the attacker's endpoint, granting the attacker control over Frank's LinkedIn account. This particular extension exhibited a low detection rate, with only one VirusTotal (VT) score, highlighting its stealth.

1. Impersonation, Dual Functionality, and Bait and Switch – DeepSeek AI / Manis AI Campaign:

This campaign illustrates a more elaborate strategy to maintain persistence and evade detection.

• Impersonation: Extensions like "DeepSeek AI Free AI Assistant" and "Manis AI Free Assistant" impersonate legitimate companies (DeepS, Manis) with convincing modern UIs and accumulate significant install bases.
• Dual Functionality: The core to their evasion is dual functionality. The extension performs its advertised, benign function—e.g., communicating with the legitimate DeepSeek API to provide AI responses—making the user unaware of any malicious activity.
• Simultaneous Exfiltration: Concurrently, the extension's background script also communicates with an attacker-controlled endpoint (e.g., api.glimmerbloop.top), exfiltrating user data. Network logs reveal requests going to both the legitimate API and the attacker's endpoint.
• Bait and Switch & Obfuscation: To further complicate detection, these extensions often employ a bait and switch tactic. They are initially published as benign versions. Later, updates introduce the exfiltration functionality. Attackers also use re-obfuscation techniques, hiding the exfiltration endpoint deep within large background script files, making manual analysis challenging. Detecting these changes often requires diffing previous versions of the extension's source code to identify newly added or modified malicious logic.

Malicious Redirection Techniques

1. Prompt Hijacking – Plexity Search Example:

This technique targets user search queries and AI prompts for monetization.

• Impersonation: An extension like "Plexity Search" impersonates Perplexity AI, leveraging similar logos and UIs.
• Vantage Point: These extensions serve as ideal vantage points for prompt data collection, extending the concept of search hijacking.
• Dual Functionality & Redirection: When a user enters a search query into the browser's omnibox, the extension captures it. It maintains dual functionality by still sending the query to the legitimate Perplexity service. However, it first redirects the request through an attacker-controlled domain before forwarding it to Perplexity.
• Abuse: This redirection allows the attacker to log the search terms and potentially manipulate the response or redirect the user to ad-based destinations, generating revenue based on the user's search intent.

1. Affiliate Fraud – Photo Room AI Editor Example:

This is a straightforward money-making strategy by abusing installation events.

• Minimal Functionality: Extensions like "Photo Room AI Editor" often have very basic, limited benign functionality (e.g., two simple photo editing features).
• API Abuse: Upon installation, the extension abuses the chrome.runtime.onInstalled API, which fires immediately. This event triggers the tabs.create API, opening a new browser tab.
• Redirect Chain: The new tab is directed to an attacker-controlled domain (e.g., photo.whatever.extends.unno), which then displays an affiliate link (e.g., promoting the Opera browser). If the user installs the advertised browser, the attacker receives an affiliate commission. This technique leverages the automatic nature of extension installation to force user exposure to revenue-generating links.

1. PUP Delivery – Pokemon Cursor Campaign:

This campaign focuses on driving traffic to grayware sites for broader monetization.

• Benign Lure: An extension like "Pokemon Cursor" offers a few benign custom cursors, attracting users with simple aesthetic modifications. The specific example cited had 200+ installs.
• Traffic Redirection: The extension's primary purpose is to act as a gateway. A link within its pop-up page, such as "get more cursors," redirects users to a grayware domain (e.g., oit.com).
• Social Engineering: This grayware site typically initiates a redirection chain, socially engineering the user into installing other potentially unwanted programs (PUPs), such as "search safely" extensions, which further facilitate affiliate fraud and other malicious activities.
• Campaign Scope: This is part of a broader campaign, with about 100 such cursor-modifying extensions collectively redirecting a significant volume of user traffic to attacker-controlled domains.

These detailed examples demonstrate how attackers meticulously craft their extensions to blend in, evade detection, and exploit both the technical capabilities and the human psychology surrounding browser extensions.

Demo / Proof of Concept

▶ Watch: Malicious redirects: Prompt hijacking and search exfiltration (20:00)

While the talk did not feature a live, interactive demonstration, the speaker effectively conveyed the proof of concept for each attack scenario through detailed explanations, architectural diagrams, and visual representations of network logs and code snippets. The methodology employed to validate the findings involved:

1. Sandbox Environment Analysis: Each malicious extension was installed and analyzed within a controlled sandbox environment. This allowed researchers to observe the extension's behavior in isolation, monitoring its interactions with web pages and external services without risk to real systems.
2. Network Log Inspection: Crucially, the talk presented evidence from network logs, illustrating how sensitive data (like email content or OTPs from the Supersonic AI example) was captured by the content script and then exfiltrated via XML fetch or HTTP requests made by the background script to attacker-controlled endpoints (e.g., API.Supersonic.email, api.glimmerbloop.top).
3. Code Review and Diffing: For "bait and switch" tactics, the speaker demonstrated the need for version diffing of the extension's source code. By comparing earlier, benign versions with later, malicious ones, researchers could pinpoint the exact code changes that introduced exfiltration logic or obfuscated endpoints. Screenshots of background script files highlighted where malicious URLs were embedded.
4. API Call Tracing: The talk detailed the specific browser APIs abused, such as chrome.runtime.onInstalled and tabs.create for affiliate fraud. The explanation walked through how these APIs were invoked within the background script to trigger redirects to affiliate links.
5. User Interface and Redirection Flow Visualization: For prompt hijacking and PUP delivery, the presentation included visuals of impersonated UIs and diagrams illustrating the redirection chains, showing how user queries first hit an attacker's domain before proceeding to legitimate services or how "get more cursors" links led to grayware sites.

This comprehensive approach, combining dynamic analysis in sandboxes with static code analysis and network monitoring, served as the proof of concept, clearly substantiating the claims of data exfiltration and malicious redirection by these GenAI extensions.

Defensive Implications

▶ Watch: Summary: GenAI extension risks and attacker tactics (25:00)

The proliferation of malicious GenAI Chrome extensions necessitates a multi-faceted defense strategy involving users, developers, and security vendors.

For Users:

• Exercise Extreme Caution: Be highly skeptical of new or unfamiliar GenAI extensions, especially those promising revolutionary features for free.
• Scrutinize Permissions: While often broad, pay close attention to the permissions an extension requests during installation. If an AI summarizer needs access to "all URLs" and "your data on all websites," it's a significant red flag.
• Verify Authenticity: Double-check the developer and source of an extension. Look for official websites, established developers, and strong community reviews, but be wary of impersonation.
• Monitor Browser Behavior: Be alert to unexpected redirects, new tabs opening without consent, or unusual network activity after installing an extension.

For Developers:

• Secure Coding Practices: Implement robust security measures, especially for message passing between content and background scripts. Validate and sanitize all data exchanged.
• Least Privilege: Request only the absolute minimum permissions necessary for your extension's functionality. Over-permissioning creates unnecessary attack surface.
• Regular Security Audits: Conduct frequent security reviews of your extension's code, especially before updates, to prevent the accidental or intentional introduction of vulnerabilities or malicious code.
• Transparency: Clearly communicate your extension's data handling practices and ensure your functionality aligns with user expectations.

For Security Vendors and Analysts:

• Enhanced Dynamic Analysis: Implement advanced sandbox environments capable of simulating user interaction and monitoring network traffic for suspicious data exfiltration (e.g., XML fetch, HTTP requests to unknown endpoints).
• Static Code Analysis and Manifest Scrutiny: Develop tools to automatically parse manifest.json files for overly broad permissions (e.g., ":///*" for host permissions, scripting API access) and analyze JavaScript code for common malicious patterns, obfuscation, and suspicious API calls.
• Behavioral Detection: Focus on detecting anomalous behaviors, such as chrome.runtime.onInstalled triggering tabs.create to redirect to affiliate domains, or content scripts sending sensitive DOM content to background scripts for external communication.
• Version Diffing: Maintain a repository of extension versions and automatically perform diffing to identify significant code changes, especially those introducing new network communication or obfuscated logic, which can signal "bait and switch" tactics.
• Impersonation Detection: Leverage machine learning and threat intelligence to identify extensions impersonating legitimate brands based on names, logos, and developer information.
• Correlation of Weak Signals: Do not dismiss extensions with a single VirusTotal detection. Correlate weak signals across multiple extensions to identify broader campaigns or emerging threats that might be under the radar of most security products.
• Threat Intelligence Sharing: Actively share Indicators of Compromise (IoCs), malicious domains, and extension IDs to foster a collective defense against these evolving threats.

By adopting these defensive strategies, the security community can better protect users from the sophisticated and often stealthy attacks perpetrated by malicious GenAI Chrome extensions.

Key Takeaways

• The explosive demand for Generative AI features has created a significant new attack surface in the browser extension ecosystem, leading to hundreds of new extensions, many of which are malicious.
• Attackers employ sophisticated techniques like impersonation, dual functionality, and bait-and-switch to exfiltrate sensitive user data, including emails, messages, and OTPs, often without explicit user interaction.
• Browser extension APIs, such as chrome.runtime.onInstalled and tabs.create, are frequently abused to perform malicious redirections for financial gain through affiliate fraud, prompt hijacking for ad revenue, and PUP delivery.
• Even seemingly benign or passively operating extensions (e.g., email summarizers) can be weaponized to silently exfiltrate highly confidential information to attacker-controlled endpoints.
• Effective defense requires a deep technical understanding of extension architecture, vigilant scrutiny of permissions, dynamic analysis in sandboxes, and meticulous code review (including version diffing to detect obfuscated changes) to uncover hidden malicious behaviors.
• Users must exercise caution, verify extension authenticity, and be aware of broad permission requests, while security vendors need to enhance behavioral detection and threat intelligence to counter these rapidly evolving threats.

About the Speaker(s)

Shresta B. Seetharam is a researcher with Palo Alto Networks, where she has been a part of the web security team for over five years. Her expertise lies in protecting enterprise customers from web-based threats, with a particular focus on malicious JavaScript detection, compromised website detection, and, more recently, browser extension security. Seetharam's research contributions are frequently shared through her LinkedIn and Twitter channels.

Mohamed Nabeel and William Melicher are also colleagues of Shresta B. Seetharam at Palo Alto Networks and contributed as co-authors to this research.