Critical SaaS, Critical Blind Spots: A Detection Engineer's Field Guide to SaaS Attacks

Critical SaaS, Critical Blind Spots: A Detection Engineer's Field Guide to SaaS Attacks

Speakers: Julie Agnes Sparks, Security Researcher, Datadog; Greg Foss, Threat Research and Detection Engineering Team Lead, Datadog

Conference: FIRSTCON

YouTube: https://www.youtube.com/watch?v=iC7XMRjPc

Overview

In the rapidly evolving landscape of cybersecurity, Software-as-a-Service (SaaS) applications have become indispensable, yet they frequently represent critical blind spots for security teams. In their FIRSTCON talk, "Critical SaaS, Critical Blind Spots: A Detection Engineer's Field Guide to SaaS Attacks," Julie Agnes Sparks and Greg Foss, both from Datadog, shed light on the emerging threat trends targeting SaaS environments and provide actionable strategies for detection engineers. The presentation dissects the common attack patterns, highlights the often-overlooked nuances of SaaS logging, and emphasizes the critical need for proactive threat hunting and robust visibility.

The core of the challenge lies in the disparity between the business criticality of SaaS applications and the often-insufficient security monitoring they receive. While cloud infrastructure logs (like AWS CloudTrail) are typically prioritized for ingestion into Security Information and Event Management (SIEM) systems, SaaS audit logs are frequently deprioritized due to cost, complexity, or a lack of understanding regarding their investigative value. This oversight creates significant opportunities for adversaries, who increasingly leverage compromised non-human identities, supply chain vulnerabilities, and sophisticated social engineering tactics to pivot from developer endpoints into critical SaaS applications, ultimately leading to widespread data exfiltration.

Sparks and Foss argue that defenders must shift their focus to understanding the intricacies of SaaS logging, recognizing the evolving attack surface, and implementing cross-platform detection capabilities. The talk serves as a field guide, offering practical advice on navigating fragmented log schemas, deciphering undocumented event types, and constructing effective threat hunting hypotheses to protect an organization's most sensitive data residing within these crucial, yet often opaque, third-party services.

Background

▶ Watch: Key emerging threat trends in SaaS attacks (2:00)

The proliferation of SaaS applications has fundamentally reshaped enterprise IT, but this convenience comes with inherent security challenges. The speakers highlight several key trends that underscore the urgency of addressing SaaS security blind spots. Foremost among these is the exponential growth of non-human identities. Greg Foss notes a metric suggesting "for every one person, they have like 80 or so different credentials," encompassing API keys, OAuth tokens, and personal access tokens (PATs). These non-human identities are identified as the number one cause of intrusions investigated by Datadog, making their compromise a primary pathway for attackers.

The OAuth ecosystem has emerged as a "new supply chain," offering an easy escalation pathway for adversaries to pivot into SaaS environments. Traditional social engineering tactics, such as help desk phishing and account provisioning scams, continue to be highly effective, often outperforming more advanced technical exploits in gaining initial access. A concerning new development is the social engineering of AI agents, as exemplified by a Meta incident where attackers tricked an AI into granting Instagram account access without proper verification. Once access is gained, the primary objective for adversaries has shifted: data exfiltration has largely replaced ransomware attacks as the most profitable endeavor. This makes the ability to detect and prevent unauthorized data egress paramount.

Supply chain compromises represent the "easiest path" into cloud applications and management planes. These attacks often originate outside the target organization's direct control, perhaps within a third-party application or a developer's environment, but invariably culminate in the compromise of critical SaaS applications that the victim organization does not directly "own." The typical attack flow described involves targeting developers through malicious IDE extensions or backdoored npm repositories, siphoning non-human identities (tokens, API keys) from their systems, and then leveraging this access to pivot into SaaS applications. From there, attackers can often escalate further into the cloud management plane, abusing API keys and even exploiting access to AI models (like AWS Bedrock) for illicit services, moving beyond traditional crypto mining to using these services "for free." Initial access vectors are diverse, including business email compromise (BEC), adversary-in-the-middle attacks, and the abuse of federated single sign-on (SSO) backdoors. Key indicators of exfiltration include "large numbers of queries" and "very short bursts of access to everything" within a SaaS application, often observed following recent security incidents.

Key Findings

▶ Watch: Common attack flow: Targeting developers for SaaS access (6:00)

The talk reveals several critical findings that underscore the current state of SaaS security and the challenges faced by detection engineers:

• Pervasive SaaS Logging Deficiencies: A central finding is the significant gap in SaaS logging capabilities compared to cloud infrastructure. Many SaaS providers offer tiered logging, where crucial investigative data (e.g., specific API calls, data exfiltration events) is locked behind more expensive "premium" plans. Speakers cited Salesforce's Event Monitoring (standard tier with 3-24 hour delays) versus Real-Time Event Monitoring (more expensive, different event types and fields) and Zendesk's audit logs often lacking visibility into data exfiltration events, as seen in the Discord breach. Furthermore, log schemas can be fragmented, inconsistent, and even contain undocumented event types, as observed with GitHub, making comprehensive detection challenging.
• Non-Human Identities as the Primary Attack Vector: The speakers consistently emphasize that non-human identities—such as API keys, OAuth tokens, and service account credentials—are the most frequently targeted assets and the leading cause of intrusions into SaaS and cloud environments. Attackers prioritize siphoning these credentials from developer endpoints, highlighting a critical focus area for defensive efforts.
• Supply Chain Attacks as a Dominant Entry Point: Compromised third-party applications and malicious software packages in developer ecosystems are repeatedly identified as significant initial access vectors. The Salesforce Drift and Clue breaches exemplify how a compromise of a trusted third-party integration can lead directly to widespread data exfiltration from customer Salesforce instances via OAuth access token abuse. Similarly, the Shy Hallude campaign demonstrates the effectiveness of backdooring npm packages to harvest credentials at scale.
• Seamless Cross-Platform Lateral Movement: Adversaries are adept at pivoting across diverse environments, from developer laptops to SaaS applications and into cloud management planes. This highlights the inadequacy of siloed security monitoring and the essential need for cross-source visibility through log normalization to track attacker activity effectively.
• Evolving and Sophisticated Attack Techniques: The threat landscape is dynamic, with attackers employing increasingly sophisticated methods. This includes the resurgence of worming and self-propagating attacks (like Shy Hallude), the use of AI-assisted phishing kits to improve social engineering efficacy, and novel credential theft methods such as siphoning sensitive tokens from the history of coding tools like Claude.

Technical Deep Dive

▶ Watch: How to prioritize threat hunting with overwhelming SaaS logs (26:00)

The technical deep dive provided by Sparks and Foss is crucial for detection engineers grappling with the complexities of SaaS security, focusing on the nuances of logging and specific attack patterns.

SaaS Logging Nuances and Challenges:

• Salesforce: The speakers meticulously detail the logging tiers. Event Monitoring is the standard, often with "3 to 24 hours" of delay. It primarily offers events like login. For real-time, comprehensive visibility, organizations must subscribe to Real-Time Event Monitoring, which is more expensive and has entirely different event types and fields. Julie Sparks notes that even with documentation, "not all the fields available that are in documentation exists." User agent logging can be inconsistent; while the standard tier sometimes extracts user agents, the more expensive tier might not, or Salesforce might replace it with internal codes like 999 (unknown) or 5238 (observed in the Clue breach), which lack public documentation. To address this, Datadog Security Labs has published a blog post offering query translations for both logging tiers across the MITRE ATT&CK framework.
• Snowflake: Snowflake logs reside in "multiple different tables," posing a complex challenge for appropriate ingestion and analysis. While direct SQL queries can be run within a Snowflake instance to access audit-related logs (e.g., client environment application field), integrating these into a SIEM allows for centralized detection and correlation. The speakers acknowledge that while they can provide "80% there" in terms of detections, the remaining "20% is really on yourself to dig in and understand your logs."
• Octa: Octa presents a unique challenge with "1,081 events" listed in its documentation. Many of these are historical or not currently in use, making it incredibly difficult for security teams to identify relevant event types for specific incidents or threat hunts. This highlights the need for focused threat modeling to prioritize which events matter most.
• GitHub: GitHub suffers from significant numbers of undocumented event types, with Datadog observing "up to like a hundred undocumented log event types." Examples include tracking sign-ins from unrecognized devices or country changes mid-session—highly valuable indicators that are not publicly documented.
• Zendesk: The Discord breach, where customer tickets were downloaded via API calls from Zendesk, underscored the necessity of premium logging tiers. Standard audit events in Zendesk are often insufficient to capture the granular API call data required to detect such data exfiltration, rendering basic logging "not useful" for incident response.

Attack Case Studies:

• Salesforce (Drift & Clue Breaches): These incidents highlight the severe impact of compromised third-party integrations. The Drift application, a Salesforce marketplace integration, was compromised, allowing attackers (attributed to UNC 363695) to use OAuth access tokens to exfiltrate "large volumes of Salesforce data" (contacts, payment methods, custom tables) from hundreds of organizations. More recently, the Clue breach followed a similar pattern: Clue, a third-party Salesforce integration, was compromised, leading to API exfiltration of data from its customers. The attackers used Python request libraries and specific user agents (e.g., 5238) and maintained a low profile with a limited number of "various hosting providers" IP addresses.
• Snowflake Incident: An info stealer deployed on a developer laptop harvested "unencrypted Snowflake credentials." This led to the compromise of "165 victims" (later confirmed to be several hundred) who lacked multi-factor authentication, resulting in massive data exfiltration and public extortion. This case emphasizes the critical link between endpoint security and SaaS compromise.
• Shy Hallude Campaign: This Team PCP campaign exemplifies self-propagating, worming attacks. Attackers harvest "anything to get into your cloud management plane, into your SaaS apps, or abuse your access to AI tools." They quickly siphon all possible information from compromised hosts, then republish backdoored npm packages. Datadog's initial scan found "over 4,400 malicious packages." The open-sourcing of Shy Hallude's toolkit has led to a proliferation of copycat attacks, making attribution more challenging.

Data Normalization:

Given the cross-platform nature of modern attacks, the speakers advocate strongly for data normalization. Using standards like OCSF or Elastic Common Schema (ECS) allows security teams to write hunting and detection queries that function across various SaaS applications and cloud infrastructure. This unified approach is critical for tracking lateral movement and accelerating detection and investigation, as adversaries rarely confine their activities to a single log source.

Demo / Proof of Concept

▶ Watch: Navigating undocumented SaaS log types and normalization (32:00)

While the speakers did not present a live demo or proof-of-concept during the talk, they extensively detailed real-world attack patterns and provided concrete examples of specific logging considerations and query types for various SaaS platforms. Their discussion focused on dissecting actual incidents (like the Salesforce Drift/Clue breaches and the Snowflake compromise) and illustrating how detection engineers can leverage available (or often unavailable) logs to identify malicious activity, rather than demonstrating a live exploit or a defensive tool in action. The emphasis was on the analytical process of threat hunting and understanding log sources.

Defensive Implications

▶ Watch: Q&A: Addressing budget constraints for critical SaaS logs (42:00)

The insights shared by Sparks and Foss carry profound defensive implications for organizations grappling with SaaS security:

• Prioritize SaaS Log Ingestion and Budget Allocation: Organizations must elevate the importance of SaaS audit logs to the same level as traditional cloud infrastructure logs. This requires budgeting for the more expensive, comprehensive logging tiers where critical security-relevant events reside, rather than being "priced out" of essential visibility. If full ingestion isn't feasible, a targeted approach based on business criticality is necessary.
• Deep Dive into SaaS Log Documentation (and its Gaps): Security teams need to proactively understand the nuances of each critical SaaS application's logging. This includes identifying available event types, understanding field definitions, acknowledging potential delays, and being aware of undocumented event types. This proactive analysis is crucial for building effective detection rules and incident response playbooks before an incident occurs.
• Implement Comprehensive Threat Modeling for SaaS: Utilize frameworks and resources (such as those from Push Security or GitHub's threat modeling documentation) to develop specific hypotheses for SaaS-related threats. Threat modeling should consider data storage, authentication mechanisms, business criticality, and current threat actor focus. This helps prioritize which SaaS applications to monitor most intensely and what specific malicious activities to hunt for.
• Focus Detections on Non-Human Identity Abuse: Given that non-human identities are the primary vector, security teams must develop robust detections specifically targeting the compromise and abuse of API keys, OAuth tokens, and service accounts. This includes monitoring for unusual access patterns, high volumes of queries, and access from previously unseen locations or user agents associated with these identities.
• Embrace Cross-Platform Detection and Log Normalization: Acknowledge that attackers will inevitably move laterally across developer endpoints, SaaS applications, and cloud environments. Implementing log normalization standards like OCSF or Elastic Common Schema (ECS) is critical to gain cross-source visibility, enabling faster correlation of events and more efficient tracking of attacker movements.
• Conduct Proactive Threat Hunting: Regular, hypothesis-driven threat hunts are essential to identify visibility gaps, validate existing detections, and uncover active intrusions. These hunts should be scoped to specific SaaS applications or attack scenarios, with clear metrics for success (e.g., new detections, identified visibility gaps, improved IR guides). This proactive stance is invaluable, as "learning Snowflake logs while doing a Snowflake incident, real rough."
• Strengthen Developer Endpoint Security: Developer workstations are frequently the initial point of compromise. Organizations must enhance security measures for these endpoints, focusing on preventing malicious IDE extensions, monitoring for backdoored packages from repositories like npm, and educating developers about the risks of sensitive data (like tokens) lingering in coding tool histories.
• Don't Blindly Trust Third-Party Tooling: Do not suppress detections or cease investigations for activity originating from service accounts or connected third-party applications. These are often the very pathways attackers exploit, as demonstrated by the Drift and Clue breaches.
• Leverage Hunt Reports for Executive Buy-in: Documenting the findings of threat hunts, including identified visibility gaps, newly created detections, and even newly discovered incidents, provides tangible evidence to justify increased investment in SaaS security and more comprehensive logging tiers.

Key Takeaways

• SaaS logs are critical but often overlooked: Organizations must recognize the immense value of SaaS audit logs for security, treating them with the same priority as cloud infrastructure logs. Proactively investigate available logs and understand their nuances, tiers, and limitations immediately.
• Non-human identities are primary targets: API keys, OAuth tokens, and service account credentials are the leading causes of SaaS intrusions. Focus detection and hunting efforts heavily on preventing and identifying the compromise and abuse of these identities.
• Lateral movement is the norm; cross-platform visibility is key: Attackers routinely pivot between developer endpoints, SaaS applications, and cloud environments. Implement log normalization (e.g., OCSF, ECS) to achieve holistic visibility and accelerate detection and investigation across your entire attack surface.
• Proactive threat hunting is non-negotiable: Conduct regular, hypothesis-driven hunts on your critical SaaS applications to identify visibility gaps, validate existing controls, and enhance your incident response readiness before an actual breach occurs.
• SaaS logging is complex and inconsistent: Be prepared for fragmented schemas, tiered pricing, delayed event delivery, and numerous undocumented event types (e.g., Salesforce, Octa, GitHub). Thoroughly understand the specific logging capabilities and limitations of each critical SaaS provider.
• Supply chain attacks pose a persistent threat: Malicious packages in developer ecosystems (like npm) and compromised third-party integrations (like Drift or Clue) are common initial access vectors. Maintain vigilance over connected applications and developer toolchains.

About the Speaker(s)

Julie Agnes Sparks is a Security Researcher at Datadog, where she works with Greg Foss on threat research. Her background includes experience on internal detection and response teams, which now informs her current role in proactive external research. Julie is deeply involved in audit log analysis, having spent significant time dissecting the intricacies of SaaS logging. She hails from Colorado and, in a fun fact, aspires to own a coffee shop in the future.

Greg Foss leads the Threat Research and Detection Engineering Team at Datadog. With approximately 15 years of experience in the security industry, he has worked for various vendors and spent many years in internal security roles. Greg's current focus is primarily on cloud security and the challenges of detecting advanced threats. He also shares his Colorado roots with Julie, making FIRSTCON a local conference for both.