IMDS Abused: Hunting Rare Behaviors to Uncover Exploits

IMDS Abused: Hunting Rare Behaviors to Uncover Exploits

Speakers: Hila Ramati, Threat Researcher, Wiz

Conference: DistrictCon

YouTube: https://www.youtube.com/watch?v=24US2pJGp40

Overview

In the dynamic landscape of cloud security, traditional signature-based detection often lags behind the pace of attacker innovation. This talk by Hila Ramati, a Threat Researcher at Wiz, in collaboration with Gili Kucinski, introduces a novel and proactive methodology for uncovering sophisticated cloud exploitation techniques, including zero-days, by focusing on anomalous behavioral patterns rather than known vulnerabilities. Titled "IMDS Abused: Hunting Rare Behaviors to Uncover Exploits," the presentation delves into how a deep understanding of normal cloud workload interactions with identity services can reveal subtle, yet critical, deviations that signal compromise.

The core of Ramati's research lies in observing how widely deployed and trusted software interacts with the Instance Metadata Service (IMDS) – a critical component for cloud identity and credential management. By analyzing cloud-scale telemetry, the Wiz research team built a "process prevalence graph" that visualizes the expected and unexpected behaviors of processes accessing IMDS. This data-driven approach allowed them to identify scenarios where common, legitimate applications suddenly requested cloud credentials when they typically never would, leading to the discovery of previously unknown vulnerabilities and even live exploitation attempts.

This methodology represents a significant shift from reactive incident response to proactive threat hunting. Instead of waiting for a vulnerability to be named or an exploit signature to emerge, the approach identifies the "flashlight app asking for contacts" moment in the cloud: an unexpected request for sensitive access by an otherwise benign process. By focusing on these rare behaviors in common software, the research successfully uncovered critical security flaws, demonstrating a powerful way to get ahead of attackers and prevent major breaches before they materialize.

Background

▶ Watch: Introduction: Unexpected identity access is a signal (0:00)

The Instance Metadata Service (IMDS) is a fundamental and ubiquitous component within modern cloud environments, serving as a small internal API accessible from within every virtual machine (e.g., AWS EC2, Azure VM, GCP Compute Engine). Its primary function is to provide workloads with temporary, short-lived credentials and configuration data, enabling instances to identify themselves and securely access other cloud resources like storage buckets, secret managers, or databases, without the need for hardcoding or storing long-term secrets directly on the instance. Cloud providers like AWS, Azure, and GCP universally rely on such metadata services, integrating them into their official SDKs, agents, and initialization scripts as the recommended and secure way for instances to authenticate and authorize actions. Consequently, IMDS access is extremely common and expected in cloud telemetry, with hundreds of processes legitimately interacting with it daily.

This very ubiquity and trusted nature make IMDS an exceptionally attractive target for attackers. While IMDS itself is not inherently insecure, its abuse typically stems from applications unknowingly acting as a proxy to it. The most common attack vector is Server-Side Request Forgery (SSRF), where an attacker tricks a vulnerable application into sending HTTP requests on their behalf to an arbitrary URL. In a traditional on-premise environment, an SSRF might grant access to an internal network service or dashboard. However, in the cloud, every instance exposes its metadata endpoint at a fixed internal IP address (e.g., http://169.254.169.254 on AWS). This means a simple SSRF vulnerability can be escalated into a direct path to temporary cloud credentials, leading to significant privilege escalation and data exfiltration.

Cloud providers have introduced several mechanisms to mitigate IMDS abuse. Azure and GCP, for instance, require a specific metadata header to be included in every request to their respective metadata services; without it, the service will not respond. AWS introduced IMDSv2, which mandates a two-step process: the instance must first make an HTTP PUT request to obtain a session token, and then include this token in subsequent HTTP GET requests to retrieve metadata. These protections are robust against many simplistic SSRF payloads that often lack control over HTTP methods or multi-step request flows. However, their effectiveness is hampered by incomplete adoption. Many legacy workloads or misconfigured environments still allow access to IMDSv1 (in AWS), which simply requires an unauthenticated HTTP GET request. Attackers, always seeking the path of least resistance, will naturally target environments where these older, less secure configurations persist.

The gravity of IMDS abuse was starkly illustrated by the Capital One breach in 2019, one of the largest cloud breaches ever recorded. In this incident, an attacker exploited an SSRF vulnerability in a web application running on AWS. Leveraging this SSRF, the attacker made HTTP calls to IMDSv1, which, at the time, was the default IMDS implementation and did not require any session tokens or authentication. This allowed the attacker to directly query the metadata endpoint and retrieve temporary IAM role credentials associated with the compromised EC2 instance. Unfortunately, these credentials were highly privileged, granting access to sensitive S3 buckets. With these keys, the attacker no longer needed access to the compromised host and could directly use AWS API calls to list and exfiltrate over 1 million customer records. This incident profoundly reshaped the industry's perspective on IMDS security, underscoring the critical importance of enforcing IMDSv2 and robust network segmentation.

Key Findings

▶ Watch: Capital One breach: Real-world IMDSv1 exploitation example (6:00)

The Wiz research team's investigation into IMDS abuse began with a critical observation: existing methods of detection were largely reactive, often surfacing IMDS abuse only during post-incident investigations. The goal was to develop a proactive, data-driven methodology capable of surfacing this entire class of abuse systematically, even in cases previously unseen, and ideally, before they escalated into full-blown breaches. Learning from past incidents, particularly the Capital One breach where a legitimate process (file) unexpectedly accessed IMDS, a simple yet powerful hypothesis emerged: "If a process that normally never talks to IMDS suddenly starts doing so, something's off."

To test this hypothesis, the researchers collected cloud-scale telemetry from thousands of diverse cloud environments, meticulously tracking which processes accessed IMDS at runtime. This extensive dataset enabled the construction of a unique analytical tool dubbed a process prevalence graph. In this graph, each column represents a process observed accessing IMDS somewhere within the dataset. The total height of the column signifies the process's overall prevalence across all monitored environments. Crucially, the color breakdown within each column differentiates behavior: pink segments indicate environments where the process did not access IMDS, while blue segments denote environments where it did.

Analyzing this graph revealed distinct patterns:

• Dominantly Blue Columns: These represent processes that are expected to interact with IMDS constantly, such as the Amazon EC2 AMI agent, cloud-init, or various cloud SDKs. Their high IMDS access prevalence is normal.
• Mixed Pink and Blue Columns: These belong to common executables like Python, Bash, or Node.js, which are used for a wide array of tasks. Some legitimate uses might require IMDS access, while others do not, resulting in a natural mix of behaviors.
• Tall Columns with a Tiny Blue Slice: This pattern emerged as the most significant anomaly signal. It signifies a widespread, legitimate process — one found in many environments — where only a very small subset of its instances ever accesses the metadata service. These are not obscure or malicious binaries but perfectly legitimate, widely used tools exhibiting highly unusual behavior.

Focusing on these "tiny blue slivers" in the prevalence graph led to remarkable discoveries. The research team found themselves asking critical questions like, "Why would a document converter or an analytics database ever need access to cloud credentials?" This line of inquiry, driven purely by behavioral anomaly, allowed them to uncover several unexpected stories, including a documented vulnerability (categorized as "expected behavior" by its maintainers), and a live, in-the-wild SSRF exploitation attempt, all without prior knowledge of specific vulnerabilities or exploit techniques. This methodology validated the approach: by establishing a robust baseline of normal cloud behavior, deviations become high-quality signals for potential compromise, enabling detection far earlier in the attack chain.

Technical Deep Dive

▶ Watch: Methodology: Building a process prevalence graph for anomalies (8:40)

The core innovation of this research lies in the creation and analysis of the process prevalence graph. This graph is generated by aggregating runtime and network telemetry data collected from thousands of cloud environments. For each observed process, the system records whether it initiates connections to the Instance Metadata Service. This data is then normalized and visualized to highlight behavioral anomalies.

Conceptually, the graph is structured with processes along the X-axis and their observed prevalence (how common they are across all monitored environments) along the Y-axis. Each process's column is segmented by color: pink indicates instances where the process was present but did not access IMDS, and blue indicates instances where it did access IMDS. This visual representation allows for immediate identification of processes whose IMDS access patterns deviate significantly from the norm. The key insight is to look for tall columns (indicating a widely deployed, common process) with only a minuscule blue sliver (indicating IMDS access in a very small percentage of its deployments).

Case Study 1: Pandoc Vulnerability

The first significant anomaly surfaced by the process prevalence graph was related to Pandoc, an open-source document converter. Pandoc is a legitimate and widely used tool found in various contexts such as CI/CD pipelines, report generators, PDF exporters, and even notebook environments. Under normal operating conditions, Pandoc has no inherent reason to interact with cloud identity services or IMDS.

However, the graph flagged one specific environment where Pandoc unexpectedly made calls to IMDS. Upon investigation, the researchers discovered that in this particular deployment, Pandoc was not being used as a command-line interface (CLI) tool but was embedded within a web service. This web service allowed users to upload HTML content for conversion into other formats, such as Markdown.

The vulnerability stems from Pandoc's handling of HTML input. HTML includes an <iframe> tag, which instructs a renderer (in this case, the server-side Pandoc instance) to fetch content from a remote URL and embed it within the document. An attacker could craft a malicious HTML input containing an <iframe> tag pointing to the internal IMDS endpoint (e.g., http://169.254.169.254 for AWS). When Pandoc processed this HTML, it would dutifully attempt to fetch the content from the specified URL on the attacker's behalf, effectively performing a classic Server-Side Request Forgery (SSRF). If the environment were running with IMDSv1, this request would directly return valid temporary IAM credentials, handing the attacker the instance's IAM role.

Interestingly, this behavior corresponds to a known vulnerability in Pandoc's HTML renderer. However, the Pandoc maintainers had categorized it as "expected behavior" rather than a flaw, addressing it by simply adding a warning to the documentation. This means that even the most up-to-date versions of Pandoc remain susceptible to this SSRF vector if embedded in a vulnerable web service.

Fortunately, in the specific environment where this anomaly was detected, a robust cloud security posture was in place: IMDSv2 was enforced. Since IMDSv2 requires an initial HTTP PUT request to obtain a session token before subsequent GET requests can retrieve credentials, the attacker's simple iframe-based GET request could not complete the necessary handshake. While this prevented a full breach, the outcome was not guaranteed, highlighting the critical role of IMDSv2 enforcement. This finding profoundly validated the methodology, as the anomaly was detected without prior knowledge of Pandoc's specific vulnerabilities, simply by observing unexpected IMDS access.

Case Study 2: ClickHouse Exploitation Attempt

The same process prevalence graph later highlighted another anomaly, this time involving ClickHouse, a widely used open-source analytics database. ClickHouse is found in numerous environments and, like Pandoc, almost never accesses IMDS under normal circumstances. The anomaly was observed in a GCP deployment, prompting the same investigative question: "What would an attacker have to do to convince ClickHouse to talk to IMDS?"

Deep diving into ClickHouse's features, the researchers identified a powerful and relevant capability: the ability for users to pull the content of a remote URL directly into a table. This feature is highly beneficial for analytics, allowing data ingestion from various external sources. However, from an attacker's perspective, this functionality acts as a potent SSRF primitive, especially in multi-tenant systems or when exposed to untrusted users.

A critical detail for exploitation within GCP (and Azure) environments is the requirement for a specific metadata header (Metadata-Flavor: Google) to be included in IMDS requests. The ClickHouse feature, crucially, not only allows control over the URL but also permits control over HTTP headers. This combination is devastating: an attacker can craft a query that instructs ClickHouse to fetch a URL pointing to the GCP metadata service (e.g., http://169.254.169.254/computeMetadata/v1/instance/service-accounts/default/token) while simultaneously injecting the required Metadata-Flavor: Google header. This allows for direct extraction of temporary credentials from the instance.

In the specific environment where this anomaly was observed, ClickHouse was running under a restricted service account, which limited the scope of credentials that could be stolen, thus mitigating the potential impact. However, the underlying vulnerability and the successful exploitation attempt were clear. This case further reinforced the methodology's effectiveness, as it surfaced a vulnerability in a completely different tool and cloud provider, again without reliance on signatures or prior intelligence. The ability to identify such subtle behavioral shifts in widely deployed software, regardless of the cloud provider or application specifics, underscores the power of this anomaly-driven approach.

Demo / Proof of Concept

▶ Watch: Case Study 2: ClickHouse SSRF vulnerability in GCP (14:35)

While the talk did not feature a live, interactive demonstration during the presentation, the speaker, Hila Ramati, presented two compelling case studies – Pandoc and ClickHouse – that served as practical proofs of concept for the methodology. These real-world examples illustrated how the process prevalence graph successfully identified anomalous IMDS access patterns, leading to the discovery of exploitable SSRF vulnerabilities and even a live exploitation attempt.

For the Pandoc case study, the "proof of concept" was the identification of a legitimate web service embedding Pandoc, allowing user-supplied HTML. An attacker could exploit this by injecting an <iframe> tag pointing to the IMDS endpoint. The speaker detailed how Pandoc, in its role as an HTML renderer, would dutifully fetch content from the IMDS URL, demonstrating the SSRF vector. The success of this "demo" in a real environment was mitigated by the enforcement of IMDSv2, which prevented the simple GET request from completing the credential acquisition.

Similarly, the ClickHouse case showcased a live SSRF exploitation attempt detected in a GCP environment. The "proof of concept" here was the discovery of a ClickHouse feature allowing users to pull remote URL content into tables, crucially with control over HTTP headers. This feature, when abused, allowed an attacker to craft a query that targeted the GCP metadata service, including the necessary Metadata-Flavor: Google header, to extract credentials. Although the impact in this specific instance was limited due to a restricted service account, it served as a clear demonstration of the vulnerability and the potential for full credential theft. Both case studies effectively functioned as concrete, real-world examples validating the efficacy of the anomaly detection methodology.

Defensive Implications

▶ Watch: Conclusion: High-quality anomaly detection from behavioral baselines (16:35)

The insights from "IMDS Abused: Hunting Rare Behaviors to Uncover Exploits" provide critical guidance for cloud defenders aiming to enhance their security posture against credential theft and SSRF-based attacks.

1. Enforce IMDSv2 and Metadata Headers Universally: This is the foundational defense. For AWS environments, IMDSv2 should be enforced on all EC2 instances. Similarly, for Azure and GCP, ensure that all metadata service requests require the specific metadata header (Metadata-Flavor: Google for GCP, X-Metadata-Request: True for Azure). These mechanisms significantly complicate simple SSRF exploitation by requiring multi-step requests or specific headers that many basic SSRF payloads cannot control. This was a critical factor in mitigating the Pandoc vulnerability.

1. Implement Principle of Least Privilege for IAM Roles/Service Accounts: Even with strong IMDS protections, a successful SSRF can still yield temporary credentials. By ensuring that IAM roles attached to instances (or service accounts for containers/VMs) have only the absolute minimum permissions required for their function, the blast radius of a compromise can be severely limited. The ClickHouse case demonstrated that a restricted service account could significantly reduce the impact of a credential theft attempt.

1. Monitor for Anomalous IMDS Access Behavior: Actively monitor and alert on any process accessing IMDS that typically does not. Establish a baseline of expected IMDS interactions within your environment. Processes like cloud-init or cloud provider agents are expected to access IMDS; however, common applications like document converters (Pandoc), databases (ClickHouse), web servers, or custom application binaries should be flagged if they suddenly start making IMDS requests. This is the core of the methodology presented and provides an early warning signal for potential exploitation.

1. Understand Application Features as SSRF Primitives: Developers and security teams must be aware of application features that can be abused as SSRF primitives. Any functionality that allows an application to fetch content from a user-supplied URL (e.g., image loading, document rendering with external content, data ingestion from remote sources, webhooks) should be treated with extreme caution. Input validation, URL allow-listing, and robust error handling are crucial to prevent these features from being weaponized.

1. Shift Towards Behavioral Anomaly Detection: Relying solely on vulnerability names, CVEs, or signature-based detection will always leave defenders behind. Adopt a proactive, data-driven approach focused on establishing a baseline of "normal" behavior and then hunting for deviations. This enables the detection of zero-days and unknown exploits by identifying the unusual actions of legitimate software, rather than waiting for known malicious signatures.

1. Regularly Audit Web Services for SSRF Vulnerabilities: Conduct regular security audits, penetration tests, and code reviews, specifically targeting web services that process user-supplied input or interact with external resources. Pay close attention to how these services handle URLs and headers, as these are common vectors for SSRF.

By combining robust preventative controls like IMDSv2 enforcement and least privilege with advanced behavioral monitoring, organizations can significantly improve their ability to detect and neutralize IMDS abuse before it leads to a major breach.

Key Takeaways

• High-quality anomaly detection in cloud environments starts with a precise understanding of how your environment is supposed to behave, making deviations a very high-quality signal.
• The Instance Metadata Service (IMDS) is a critical cloud component often targeted by attackers via Server-Side Request Forgery (SSRF) vulnerabilities to steal temporary cloud credentials.
• While cloud providers offer protections like IMDSv2 (AWS) and mandatory metadata headers (Azure/GCP), incomplete adoption or specific application features can still leave environments vulnerable.
• Focusing on rare behavioral patterns in common, legitimate software (e.g., a document converter or database unexpectedly accessing IMDS) can uncover previously unknown vulnerabilities and live exploitation attempts, including zero-days.
• A process prevalence graph, built from cloud-scale telemetry, provides a powerful visual and analytical tool to identify these subtle, yet critical, anomalies without relying on traditional signatures or vulnerability intelligence.
• Proactive behavioral monitoring allows defenders to detect threats much faster, often before they are named or widely understood, enabling a security posture that moves faster than attackers.

About the Speaker(s)

Hila Ramati is a Threat Researcher at Wiz, a leading cloud security company. Her work focuses on uncovering novel attack techniques and vulnerabilities within cloud environments. This presentation is based on a collaborative research project with Gili Kucinski, who is also a part of the Wiz research team. Together, their research highlights innovative methods for proactive threat hunting in the cloud.