Breaking ML Services: Finding 0-days In Azure Machine Learning

Breaking ML Services: Finding 0-days In Azure Machine Learning

Speakers: Nitesh Surana, Cloud and Container Threat Researcher, Trend Micro

Conference: HITB Phuket

YouTube: https://www.youtube.com/watch?v=-K08hpzevYY

Overview

In this insightful talk from HITB Phuket, Nitesh Surana, a Cloud and Container Threat Researcher at Trend Micro, delves into critical security vulnerabilities discovered within Azure Machine Learning (AML), Microsoft's cloud-based Machine Learning as a Service (MLaaS) offering. The presentation, titled "Breaking ML Services: Finding 0-days In Azure Machine Learning," highlights a series of zero-day vulnerabilities ranging from clear-text credential logging to sophisticated persistence mechanisms leveraging compromised internal service certificates. Surana's research exposes how seemingly benign configurations and deeply embedded internal agents can create significant attack surfaces, enabling attackers to gain persistent access to sensitive data and even Azure AD tokens from non-Azure environments.

The talk is particularly relevant given the rapid adoption of cloud-based AI and machine learning platforms, a trend predicted by Gartner to increase fivefold by 2023. As companies increasingly rely on services like AML to train large language models (LLMs) and develop AI-powered products, understanding the underlying security posture of these platforms becomes paramount. Surana's findings underscore the often-overlooked risks associated with managed cloud services, where internal components and default behaviors can introduce "silent threats" that users are largely unaware of.

This article provides a comprehensive breakdown of Surana's discoveries, detailing the technical intricacies of each vulnerability, its impact, and the broader implications for cloud security and the shared responsibility model. It serves as a crucial resource for defenders, cloud architects, and machine learning practitioners seeking to fortify their environments against advanced threats targeting MLaaS platforms.

Background

▶ Watch: Beginning the Azure ML security research journey (2:30)

The increasing reliance on cloud-based machine learning platforms, driven by the demand for AI innovation and the rise of technologies like ChatGPT, has brought services like Azure Machine Learning into the spotlight. AML provides a comprehensive environment for MLOps, allowing data scientists to manage their entire machine learning lifecycle, from data preparation and model training to deployment and monitoring, directly from a browser, VS Code, or SDKs.

At its core, AML operates through workspaces, which serve as centralized hubs for all ML activities. These workspaces leverage a mix of user-managed and Microsoft-managed services. Key user-managed dependencies include storage accounts (for notebooks, datasets, models, logs), Key Vaults (for secrets), Container Registries (for environment images), and Application Insights (for model monitoring). For executing ML tasks, AML offers various compute targets, such as compute clusters (managed VM clusters) and Kubernetes clusters. A central focus of Surana's research was the compute instances, which are managed Ubuntu-based virtual machines residing in a Microsoft subscription. These instances act as development environments, pre-configured with tools like Jupiter, VS Code, Docker, and Python, where users run their code and perform model training.

A critical component for data access within AML is the concept of data stores. These are references to a workspace's storage account, designed to securely access file shares or blob storage without hardcoding access keys or SAS credentials in scripts. By design, data stores in AML support credential-based authentication for file shares, using the storage account name as the username and the storage account's primary access key as the password. Furthermore, a common file share is mounted across all compute instances within a workspace, allowing multiple users to access each other's files—a design choice that, while not a vulnerability in itself, creates a shared risk surface. The default user on compute instances, azureuser, is configured to assume root privileges via sudo without requiring a password, further simplifying privilege escalation within the instance.

The research also touched upon Azure Batch, a service for running large-scale workloads. AML compute instances utilize elements of Azure Batch, including concepts like nodes (VMs), pools (groups of nodes), jobs (collections of tasks), and tasks (individual command runs). Crucially, start tasks in Azure Batch run when a node starts up, and their outputs (including standard error and standard out logs) are stored in specific paths (mount/batch/tasks/startup). This mechanism proved to be a significant initial entry point for discovering credential leakage.

Surana's investigation was sparked by a broader trend of vulnerabilities being reported in Jupiter Notebook-based services across various cloud providers (Azure, AWS, GCP), suggesting a common underlying vulnerability pattern or misconfiguration. This led him to explore Azure's Jupiter-enabled services, eventually focusing on Azure Machine Learning.

Key Findings

▶ Watch: User JWT tokens found in Nginx access logs (17:40)

Nitesh Surana's research unveiled several critical zero-day vulnerabilities in Azure Machine Learning, collectively demonstrating a significant compromise potential for customer data and Azure environments. These findings can be categorized into three main areas:

1. Clear-Text Credential Logging and Storage (CVE-2023-28267):

• Storage Account Access Keys: The primary access keys for Azure storage accounts were found to be logged in clear text within the standard error logs of Azure Batch start tasks (standard error.text files located in mount/batch/tasks/startup). These keys were part of sudo mount commands executed by Microsoft-managed processes. Furthermore, the storage account primary access key was also stored in clear text within environment variables (pass WD) used by critical internal agents, specifically the DSi Mount Agent and DSi Idle Stop Agent, located in mount/batch/tasks/startup/working_directory/DSi.
• Jupiter User JWTs: JSON Web Tokens (JWTs) belonging to users accessing compute instances via Jupiter Lab or the browser-embedded terminal were logged in clear text within the Nginx proxy access logs. These tokens were passed as URL parameters during GET requests, a practice explicitly allowed by Jupiter's documentation but dangerous in default logging configurations.

1. Unauthenticated Information Disclosure via DSi Mount Agent (MLC Bug):

• A critical internal agent, the DSi Mount Agent, running as root on compute instances, exposed an unauthenticated API on port 46802 within the virtual network. Through reverse engineering, Surana discovered endpoints that allowed network-adjacent attackers to list installed services (/services) and, more critically, view the systemd logs of any service (/logs). This MLC bug enabled an attacker to observe commands executed by data scientists using Jupiter terminals, including sensitive sudo commands (e.g., sudo cat /etc/shadow), potentially revealing further credentials or system configurations if logged by the service.

1. Persistent Access and Lateral Movement via Compromised Internal Certificates and Keys:

• Surana discovered that specific internal certificates and private keys, located at mount/batch/tasks/startup/certs/, were used by internal agents like the identityresponderd daemon and the DSi Mount Agent to communicate with various Azure ML internal endpoints (referred to as XDS endpoint).
• Storage Account Persistence: If these certificate and key pairs, along with specific environment variable files (DSi_Mount_agent.env, DSi_Idle_Stop_agent.env) containing an encrypted symmetric key, were compromised from a compute instance, an attacker could use them from a non-Azure environment to decrypt a JWE (JSON Web Encryption) obtained from the XDS endpoint. This JWE contained the storage account primary access key. Crucially, this attack vector allowed an attacker to persistently retrieve even rotated storage account keys, rendering key rotation ineffective as a sole remediation.
• Azure AD Token Compromise: Further investigation revealed that these same compromised certificates and keys could be used to fetch Azure Active Directory (Azure AD) tokens for both System-Assigned Managed Identities and User-Assigned Managed Identities assigned to the compute instance, again from non-Azure environments. This directly contradicted Microsoft's documentation, which states that only the assigned Azure resource can use the identity to request tokens, thus creating a significant privilege escalation and lateral movement opportunity.
• Evasive Detection: Logs generated by malicious use of these certificates and keys were found to be almost identical to legitimate activity, crucially lacking source IP information, making detection extremely challenging. The certificates were valid for two years, and the only way to invalidate them was to delete the entire compute instance.

These findings highlight a systemic issue where internal cloud agent configurations and default logging practices created significant security gaps, enabling various forms of credential leakage, unauthenticated information disclosure, and persistent access to critical cloud resources.

Technical Deep Dive

▶ Watch: Bypassing storage account key rotation with compromised certs (43:30)

The technical depth of Surana's research is best understood by dissecting each "chapter" of his journey, revealing the intricate details of discovery and exploitation.

Chapter 1: Data Scientist is Looking for Their Keys (Credential Logging)

The initial phase of research focused on understanding how Azure Machine Learning instances operate, particularly regarding file share mounting. Surana observed that Azure file shares and data stores use credential-based authentication, prompting questions about how these credentials were handled during the mounting process.

By examining the directory structure on a newly created compute instance, specifically mount/batch/tasks/startup, the researchers identified standard error.text and standard out.text files. These files contained logs from Azure Batch start tasks, which are Microsoft-managed scripts executed when a node (in this case, the compute instance VM) starts up. Within the standard error.text logs, they discovered sudo mount commands that explicitly logged the storage account name (username) and the storage account's primary access key (password) in clear text. This direct logging of highly sensitive credentials was a critical finding.

Upon reporting this, Microsoft implemented a fix that masked the password field in these specific sudo mount commands. However, this led Surana to investigate other potential clear-text credential storage. Further exploration of the mount/batch/tasks/startup/working_directory/DSi directory, where DSi agents (internal Microsoft-managed agents for diagnosis and monitoring) reside, revealed configuration files that supplied environment variables to these agents. Specifically, the DSi mounting agent and DSi idle stop agent were found to contain the storage account's primary access key in an environment variable named pass WD. This indicated a more widespread issue of credentials being stored in clear text within the compute instance's environment.

Chapter 2: Wait is That My Token (Jupiter Token Leak)

The next area of investigation targeted user access to compute instances, particularly through browser-based interfaces like Jupiter Notebook, Jupiter Lab, and the terminal. These requests are proxied to the compute instance via a load balancer. The URL structure revealed an Nginx proxy listening on a fixed port, 44224, with SSL enabled and requiring client-side certificates.

Using TCP dump to capture network traffic on the compute instance, Surana observed that when a user terminated a Jupiter terminal session, a GET request followed by a DELETE request was sent. Crucially, the GET request contained a token parameter in the URL, which held the JWT of the user accessing the Jupiter instance. This token was Base64 encoded, but easily decodable. The Nginx access logs on the compute instance also captured these full URLs, thus logging the user's JWT in clear text.

Further research into Jupiter's documentation confirmed that providing the authentication token as a URL parameter is a valid method. While functionally correct, logging this sensitive information in clear text within Nginx access logs represented a significant security oversight. The compromised JWT could be used to impersonate the user and access their resources within the AML workspace.

The combination of clear-text storage account keys and user JWTs posed a substantial risk:

• Developer Exposure: Developers sharing error or debug logs (e.g., on GitHub) unknowingly expose these credentials.
• Dependency Confusion/Supply Chain Attacks: Malicious code executed via dependency confusion attacks could easily exfiltrate these credentials from files or environment variables.
• Workspace Compromise: Compromising a single file share or compute instance via these leaked credentials could lead to the trivial compromise of the entire storage account and, consequently, the entire AML workspace, impacting all users within it.

Microsoft addressed these issues with CVE-2023-28267, ensuring that the storage account access key is no longer logged or stored in clear text and that user JWTs do not reach the compute instance for logging.

Chapter 3: Spying the Scientist (MLC Bug)

This chapter explored vulnerabilities within virtual network (VNet) environments, a common deployment scenario for AML where training environments need private access to other Azure services. The scenario considered was a compromised Azure VM within the same VNet as a compute instance.

An nmap scan from the compromised VM against the compute instance revealed an open port 46802. Investigation showed that a process named DSi Mount Agent, running as root, was listening on this port. Since the DSi Mount Agent binary was not stripped, Surana used IDA Pro to reverse engineer it. The analysis revealed a function named DSiStartAPIService that exposed several unauthenticated API endpoints over the network.

Key unauthenticated endpoints discovered included:

• /sync: Executes a sync command on a file (not immediately exploitable for RCE).
• /mount: Remounts the Azure file share.
• /os_image: Views the OS image version.
• /services: Lists installed services and their statuses.
• /logs: Views the logs of any installed service.

The /services and /logs endpoints were particularly interesting. A network-adjacent attacker could list all services and then request the logs for any of them without authentication. During a demonstration, Surana showed how requesting the logs for the jupyter service revealed commands executed by the data scientist through the Jupiter terminal, including sudo commands like sudo cat /etc/shadow. This MLC bug (Machine Learning Compromise) allowed an attacker to "spy" on the data scientist's actions, potentially capturing sensitive commands or data that might be logged. Microsoft fixed this by restricting the port exposure, making it only reachable via the Nginx proxy and requiring client-side certificates.

Chapter 4: Can You Really See Me (Persistent Access via Certificates)

The final and most critical phase of the research focused on managed identities, a recommended secure way for Azure resources to access other services without hardcoding credentials. Managed identities work by allowing a resource to request an Azure AD token for a target service.

Surana investigated the az login --identity command execution on a compute instance, again using TCP dump. This revealed outbound GET requests to a local IP address and port, which was identified as the identityresponderd daemon (Azure Batch AI Identity Responder Daemon). This daemon fetches environment variables from /etc/environment.sso, including MSI_ENDPOINT and MSI_SECRET.

Further analysis revealed that the final request for an Azure AD token was a POST request to region.cert.api.azureml.ms using a specific certificate and private key pair located at mount/batch/tasks/startup/certs/. Initially, attempts to use these certificates from a non-Azure environment resulted in a 401 Unauthorized error, suggesting they were tied to the specific compute instance.

However, revisiting the DSi Mount Agent (from Chapter 3), Surana discovered that it also communicates with an XDS endpoint (defined by the XDS_ENDPOINT environment variable) using the same certificate and private key pair. This agent's primary role is to ensure the Azure file share remains mounted.

Deep diving into the DSi Mount Agent binary, a function called get_workspace_secrets was found. By crafting a POST request with request_type: get_workspace_secrets to the XDS endpoint using the compromised certificate and private key, the researchers received a response containing the storage account name and an encrypted JWE. With the help of David, the decryption process was reverse-engineered. It involved using two environment variables from the DSi_Mount_agent.env and DSi_Idle_Stop_agent.env files: AZ_LS_ENCRYPTED_SYMMETRIC_KEY and a private key. Decrypting the symmetric key first, then using it to decrypt the JWE, yielded the storage account's primary access key.

This led to a critical finding: if an attacker compromised the certificate, private key, and relevant environment variable files from a compute instance, they could use these from a non-Azure environment to persistently fetch the storage account's primary access key. A live demo showed that even after the storage account key was rotated in Azure, the attacker could still retrieve the newly rotated key, demonstrating a significant persistence mechanism.

Beyond storage account keys, the research also identified other functions within agents like hosttools, such as get_azure_ad_token. By crafting a POST request with request_type: get_aad_token and specifying a resource (e.g., for Azure Key Vault or Azure Storage), the compromised certificate and key allowed an attacker to fetch Azure AD tokens for System-Assigned Managed Identities from a non-Azure environment. A bonus finding showed that by specifying the client_id in the request body, User-Assigned Managed Identity tokens could also be fetched. This directly violated the documented security principle that only the assigned Azure resource should be able to request tokens, opening avenues for lateral movement and privilege escalation from outside the Azure environment.

A concerning aspect of this persistence was the logging: legitimate and malicious uses of these certificates generated almost identical logs, critically lacking source IP information, making detection extremely difficult. The certificates were valid for two years, and the only way to revoke them was to delete the entire compute instance.

Demo / Proof of Concept

▶ Watch: Fetching managed identity tokens from non-Azure environments (48:00)

Surana punctuated his technical explanations with compelling demonstrations, showcasing the real-world impact of the discovered vulnerabilities.

1. MLC Bug Demo (Spying on the Data Scientist):

• This demonstration illustrated the MLC bug (unauthenticated information disclosure via the DSi Mount Agent).
• Setup: A data scientist on a compute instance executed a sensitive command, sudo cat /etc/shadow, within a Jupiter terminal.
• Attacker Action: From a network-adjacent Azure virtual machine, an attacker sent an unauthenticated request to the DSi Mount Agent's API on port 46802 to retrieve the logs of the jupyter service.
• Result: The Jupiter service logs, displayed on the attacker's machine, clearly showed the sudo cat /etc/shadow command that the data scientist had executed. This demonstrated how a network-adjacent attacker could silently monitor user activities and potentially capture sensitive information or commands, even those run with sudo.

1. Persistent Storage Account Access Demo:

• This demo highlighted the ability of an attacker to maintain persistent access to Azure storage account keys even after they are rotated.
• Setup: Surana, acting as an attacker, demonstrated using a tool from his laptop (simulating a non-Azure environment, though using GitHub Codespaces which is Azure-based, the principle of being outside the compute instance was key) to fetch the initial primary access key of an AML workspace's storage account. This was achieved by using the compromised certificate, private key, and environment variable files obtained from a compute instance.
• Remediation Attempt: He then navigated to the Azure portal, rotated the storage account's primary access key, and synchronized the workspace to use the new key.
• Attacker Persistence: Immediately after the key rotation, the attacker's tool was run again from the non-Azure environment.
• Result: The tool successfully fetched the newly rotated storage account primary access key. This conclusively proved that simple key rotation is insufficient to mitigate compromise if these specific internal certificates and keys are exfiltrated, as they allow for the programmatic retrieval of current keys.

1. Managed Identity Token Compromise Demo:

• This demonstration showcased the ability to fetch Azure AD tokens for managed identities from a non-Azure environment, bypassing security principles.
• Setup: Two compute instances were used, one with a System-Assigned Managed Identity and another with a User-Assigned Managed Identity. The necessary compromised certificate, private key, and environment variable files were collected from each instance.
• Attacker Action: From a non-Azure environment (GitHub Codespaces), Surana used a Python script (the p script in his demo) to target the XDS endpoint using the respective compromised files.
• Result: The script successfully fetched the Azure AD JWT for the System-Assigned Managed Identity and subsequently for the User-Assigned Managed Identity. This demonstrated a direct violation of the principle that only the Azure resource itself should be able to request these tokens, enabling an attacker to assume the identity's permissions and potentially move laterally or escalate privileges within the Azure environment.

These demos vividly illustrated the severity of the discovered vulnerabilities, emphasizing the critical need for enhanced security measures and better logging within Azure Machine Learning environments.

Defensive Implications

▶ Watch: Q&A on cross-environment access with stolen certificates (64:40)

The findings from Nitesh Surana's research into Azure Machine Learning carry significant implications for cloud defenders, developers, and cloud architects. Understanding these vulnerabilities is crucial for implementing effective defensive strategies.

1. Prioritize Secure Logging and Monitoring: The most immediate implication is the absolute necessity of robust logging. Cloud service logging must be enabled, and organizations need to understand what is being logged and what is not. The lack of source IP information in certificate usage logs, for instance, crippled detection capabilities. Defenders should:

• Review default logging configurations: Do not assume defaults are secure. Explicitly configure logging to capture sensitive events, including source IPs for requests involving internal agents or managed identities.
• Leverage Cloud-Native Solutions: Utilize Azure Monitor, Azure Sentinel, and other cloud-native logging/SIEM solutions to centralize, analyze, and alert on suspicious activities.
• Understand Log Contents: Educate developers and security teams on the potential for credentials or sensitive information to appear in logs, especially debug or error logs, to prevent inadvertent public exposure.

1. Strict Adherence to Least Privilege: The compromise of managed identities highlighted the dangers of over-privileged identities.

• Scope Identities Carefully: When assigning System-Assigned Managed Identities or User-Assigned Managed Identities, grant only the absolute minimum permissions required for the resource to function.
• Regularly Review Permissions: Periodically audit the roles and policies assigned to managed identities to ensure they align with least privilege principles.

1. Enhanced Threat Modeling for Managed Services and Agents: The discovery of vulnerabilities in internal agents (DSi Mount Agent, identityresponderd) underscores a gap in traditional threat modeling.

• Assume Breach Philosophy: Approach cloud environments with an "assume breach" mindset. What happens if a compute instance is compromised, and internal files (like certificates, private keys, environment variables) are exfiltrated?
• Focus on Agent Features: Conduct specific threat modeling exercises for internal cloud agents, considering their functionality, exposed APIs (even if internal), and how they handle sensitive data. Users are often unaware of these "secret agents" and their potential attack surface.

1. Implement Defense-in-Depth with Network Isolation: While the MLC bug was fixed, the principle of network segmentation remains vital.

• Virtual Networks, Private Links, Private Endpoints: Always deploy AML workspaces and compute instances within Virtual Networks (VNets), utilizing Private Links and Private Endpoints to ensure traffic remains within the Azure backbone and is not exposed to the public internet or even other internal VNet segments unless explicitly allowed. This significantly reduces the attack surface for network-adjacent threats.
• Zero Trust Principles: Do not automatically trust internal network traffic. Implement Zero Trust principles, verifying every request regardless of its origin within the network.

1. Secure Development Practices and Configuration Review:

• Open-Source Tool Configuration: When using open-source tools like Jupiter or Nginx, review their default configurations carefully. Avoid logging sensitive information, especially when passed via URL parameters. Daniel Mier has long advocated against placing sensitive information in URL parameters due to unknown inline components.
• Integrity Verification: Implement mechanisms to verify the integrity of Jupiter notebooks, scripts, models, and datasets to detect unauthorized changes or malicious injections.

1. Understand Shared Responsibility and Invalidation:

• Cloud Provider vs. Customer: These vulnerabilities fell squarely on the cloud provider's side, emphasizing that even "managed" services can harbor critical flaws. However, customers are responsible for configuring and monitoring their environments securely.
• Certificate Invalidation: The discovery that compromised certificates were valid for two years and could only be invalidated by deleting the entire compute instance highlights a severe operational challenge. Organizations must be prepared for this drastic measure in the event of compromise and have robust backup and recovery strategies in place.

1. Leverage ML-Specific Threat Frameworks:

• Atlas Framework: Utilize frameworks like the Adversarial Threat Landscape for Artificial Intelligence Systems (Atlas) framework (MITRE ATT&CK for ML) to understand and model potential threats specific to machine learning environments, aiding in proactive defense and incident response planning.

By adopting these defensive strategies, organizations can significantly reduce their exposure to the types of sophisticated, persistent threats uncovered in this research, moving towards a more secure cloud machine learning posture.

Key Takeaways

• Silent Threat of Clear-Text Credentials: Critical credentials, including storage account primary access keys and user JWTs, were found logged or stored in clear text within Azure Machine Learning environments. These "silent threats" are often unknown to users and can be easily exfiltrated by attackers, leading to widespread compromise.
• Hidden Attack Surface in Managed Services: Internal Microsoft-managed agents (e.g., DSi Mount Agent, identityresponderd) exposed unauthenticated APIs and utilized internal certificates and keys, creating an invisible attack surface. Vulnerabilities in these agents can lead to unauthenticated information disclosure and unauthorized access.
• Persistent Access Beyond Key Rotation: Compromised internal certificates and private keys, combined with specific environment variables, enabled attackers to persistently fetch even rotated storage account access keys from non-Azure environments, rendering traditional key rotation ineffective as a standalone defense.
• Managed Identity Bypass for Lateral Movement: The research demonstrated that Azure AD tokens for managed identities could be fetched from non-Azure environments using compromised certificates and keys, directly contradicting Microsoft's documented security principles. This opens significant avenues for lateral movement and privilege escalation within the Azure ecosystem.
• Inadequate Logging Hinders Detection: Logging for critical activities, such as managed identity token requests via compromised certificates, was found to be almost identical between legitimate and malicious actions, crucially lacking source IP information. This severely impairs detection capabilities and allows attackers to operate evasively.
• Defense-in-Depth and Zero Trust Are Paramount: The findings underscore the critical need for a defense-in-depth strategy, including strong network isolation (VNets, Private Links), strict adherence to least privilege for all identities, and a Zero Trust approach where internal traffic is never implicitly trusted.

About the Speaker(s)

Nitesh Surana is a Cloud and Container Threat Researcher at Trend Micro. His work primarily focuses on uncovering security vulnerabilities and enhancing threat intelligence within cloud and containerized environments. Surana was recognized for his contributions to security research by ranking 75th on the Microsoft MSRC MVR 2023. With approximately 2.5 years of experience in the threat research space, he possesses expertise in security operations, threat hunting, and vulnerability disclosures. He is also an active member of null, an open security community with a global presence. Nitesh's passion for exploits began early, writing his first one in 2009 for a game called "Spider Solitaire," where he used "Cheat Engine" to modify game steps. He humorously noted writing his first song in 2018, long after his first exploit. A keen adventurer, he once tracked 4 kilometers with a minor ACL tear, showcasing his persistence both in research and in life.