Inside Microsoft's Battle Against Cloud-Enabled Deepfake Threats

Inside Microsoft's Battle Against Cloud-Enabled Deepfake Threats

Speakers: Alessandro Brucato; Stefano Chierici

Conference: fwd:cloudsec North America 2025

YouTube: https://www.youtube.com/watch?v=EDQfmtwa49A

Overview

Alessandro Brucato and Stefano Chierici present an investigation into how attackers abuse stolen cloud API credentials to generate deepfakes at scale, connecting the dots between LLMjacking (unauthorized use of cloud AI APIs via stolen credentials), the OAI Reverse Proxy ecosystem, a purpose-built deepfake generation tool called D3U, and Microsoft's December 2024 lawsuit against the individuals behind these tools. The talk maps the full attack chain from credential theft through proxy infrastructure to deepfake generation, examines how D3U bypasses Azure OpenAI's content safety controls by jailbreaking prompts and stripping C2PA content credential metadata, and reviews Microsoft's AI shared responsibility model and the technical defenses available within Azure OpenAI.

Background

▶ Watch: Deepfake case studies: $25M fraud at Arup and election manipulation in Slovakia (1:30)

Deepfakes -- synthetic media generated by AI to impersonate real people -- have moved from novelty to operational threat. The speakers cite three representative incidents: the Mr. Deepfake website (shut down after offering on-demand deepfake content for any person), the Arup case (a British engineering company tricked into sending $25 million to attackers using deepfake video of senior management in a call), and election manipulation in Slovakia (a deepfake video of a top candidate claiming to have rigged the election).

LLMjacking is the practice of stealing cloud API credentials and using them to make unauthorized calls to AI services. The speakers have researched this for approximately two years and were among the first to publish on it. Attackers obtain credentials by exploiting vulnerable applications, scraping exposed secrets from repositories, or stealing them from compromised environments. These credentials are then used to call cloud AI APIs -- Azure OpenAI, AWS Bedrock, Vertex AI -- with the account owner bearing the cost. LLMjacking is now included in the MITRE ATT&CK framework under resource hijacking.

The OAI Reverse Proxy is a key piece of infrastructure in the LLMjacking ecosystem. Attackers operate instances of this proxy, pre-loaded with stolen credentials for multiple cloud AI services, and sell access to end users. The proxy supports all major cloud AI APIs and can automatically provision Cloudflare domains to mask the real IP address. This creates a turnkey service where buyers can generate AI content without needing their own credentials or technical knowledge.

Key Findings

▶ Watch: OAI Reverse Proxy: turnkey infrastructure for selling stolen AI access (6:00)

1. Microsoft's December 2024 lawsuit identified key actors. The lawsuit named individuals connected to the OAI Reverse Proxy ecosystem, including its creator and developers who built extensions for specific cloud providers (GCP, Azure). The speakers' own research, published in February 2025, corroborated Microsoft's findings. One threat actor, identified as "Drago," was observed selling credential theft and validation scripts on Patreon alongside operating LLMjacking infrastructure.

1. D3U is the primary deepfake generation tool in this ecosystem. D3U is a Python-based web UI for generating images using OpenAI's DALL-E 3 model. It integrates with the OAI Reverse Proxy for authentication, meaning users only need proxy access rather than their own API keys. The tool was publicly available at a wry.org URL (now taken down) and its GitHub repository has been removed.

1. D3U bypasses Azure's content safety controls in two ways. First, it includes a jailbreaking option that wraps the user's prompt in a jailbreak template instructing the LLM not to modify the prompt before passing it to DALL-E 3, circumventing Azure's prompt transformation system that enriches or sanitizes prompts containing flagged content. Second, it strips C2PA content credentials metadata -- Azure's mechanism for embedding provenance information in AI-generated images that allows anyone to verify an image was AI-generated. By removing this metadata, D3U makes deepfakes harder to identify as synthetic.

1. The attack chain is accessible and scalable. An end user with no technical expertise can buy access to an OAI Reverse Proxy instance, use D3U's web interface to generate deepfakes with jailbroken prompts, and receive images with stripped provenance metadata. The entire workflow leverages someone else's stolen cloud credentials and someone else's infrastructure.

Technical Deep Dive

▶ Watch: D3U tool: jailbreaking Azure prompts and stripping C2PA metadata (10:00)

The full attack chain operates as follows:

1. Credential acquisition: Attackers scan for exposed cloud credentials in public repositories, exploit vulnerable applications, or purchase stolen credentials. The focus is on credentials with access to Azure OpenAI, AWS Bedrock, or other cloud AI services.
2. Proxy deployment: Stolen credentials are loaded into OAI Reverse Proxy instances. The proxy automatically handles API routing across cloud providers and can provision Cloudflare domains for operational security.
3. Access monetization: Proxy operators sell access tokens or subscriptions to end users, creating a layered criminal ecosystem where the credential thief, proxy operator, and content generator can be different actors.
4. Content generation: End users connect D3U to the proxy and submit prompts. D3U's jailbreaking feature wraps prompts in a template that instructs the model to bypass Azure's prompt revision system. The generated images have their C2PA metadata stripped before delivery.

Microsoft's AI shared responsibility model divides obligations across three areas. AI Usage (mostly customer responsibility) covers staff training, access management, and data governance. AI Application (shared responsibility) covers application design, AI plugin interaction, and content safety guardrails. AI Platform (mostly Microsoft responsibility) covers compute infrastructure and base model integrity.

Azure OpenAI's technical defenses include content filtering using classification models on both input and output, abuse detection heuristics triggered by anomalous usage patterns, prompt transformation to sanitize harmful content, prompt shields against adversarial jailbreaking, and 30-day prompt storage with human review capability for flagged content.

For runtime detection, Azure OpenAI does not natively log LLM usage events. Organizations must configure Azure API Management as a wrapper around Azure OpenAI to capture API calls. From there, events can be sent to streaming services like Event Hub and forwarded to third-party detection platforms. Detection rules should monitor the image/generations/create API endpoint and analyze prompt content and LLM responses using KQL queries.

Demo / Proof of Concept

▶ Watch: Microsoft AI shared responsibility model breakdown (14:00)

No live demo was presented. The speakers showed screenshots from actual LLMjacking incidents they observed in monitored environments, including the identification of the Drago threat actor, D3U's jailbreaking prompt in API logs, and the workflow for fingerprinting attacks back to specific tools and threat actors.

Defensive Implications

▶ Watch: Runtime detection: configuring Azure API Management for LLM monitoring (17:30)

Organizations using Azure OpenAI should implement several layers of defense:

• Credential hygiene is the first line of defense. LLMjacking begins with stolen credentials. Rotate API keys, monitor for exposed credentials in repositories, and prefer role-based access over API keys for AI service authentication.
• Configure Azure API Management around Azure OpenAI endpoints to enable logging of all API calls. Without this, you have no visibility into LLM usage in your environment.
• Build detection rules for suspicious image generation API calls, anomalous usage volumes, and known jailbreaking prompt patterns.
• Restrict AI applications to private networks and limit access to approved network subnets.
• Leverage advanced content guardrails that apply to generated images, not just text.
• Monitor for C2PA metadata stripping. While you cannot prevent external tools from stripping metadata, awareness of this technique is important for verifying the provenance of images your organization encounters.
• Policy and regulatory engagement matters. Microsoft's report to policy makers emphasizes the need for regulations requiring AI-generated content labeling. Organizations should track and prepare for these requirements.

Key Takeaways

• LLMjacking -- stealing cloud credentials to abuse AI APIs -- is an established attack pattern now recognized in the MITRE ATT&CK framework, with deepfake generation as a primary use case.
• The OAI Reverse Proxy and D3U tools create a turnkey ecosystem where non-technical users can generate deepfakes at scale using stolen cloud credentials.
• D3U bypasses Azure OpenAI's content safety controls through prompt jailbreaking and C2PA metadata stripping, making deepfakes harder to detect as synthetic.
• Azure OpenAI does not natively log LLM usage; organizations must configure Azure API Management to gain visibility into AI API calls.
• Microsoft's December 2024 lawsuit identified key individuals in the LLMjacking ecosystem, with corroborating evidence from the speakers' independent research.
• Defending against this threat requires a combination of credential management, API monitoring, content safety guardrails, network restrictions, and industry-level regulatory action.

About the Speaker(s)

Stefano Chierici and Alessandro Brucato (referred to as "Bruce") are security researchers who have spent approximately two years investigating LLMjacking attacks across AWS, Azure, and GCP environments. They were among the first to publish research on LLMjacking and have developed methodologies for fingerprinting LLMjacking attacks by analyzing API logs, user agents, and prompt patterns. They created an attack status model for AWS Bedrock and have tracked the evolution of the threat actor ecosystem from credential theft through proxy monetization to content generation.