From Lure to Alert: Implementing Canary Tokens in M365

Ryan O'Donnell (Senior Security Engineer · Microsoft)

Wild West Hackin' Fest @ Mile High 2026 · Day 1

Overview

In his Wild West Hackin' Fest @ Mile High talk, "From Lure to Alert: Implementing Canary Tokens in M365," Ryan O'Donnell, a Senior Security Engineer at Microsoft, delves into an innovative approach for early detection of post-compromise activity within Microsoft 365 environments. The presentation addresses the critical shift in attacker focus from traditional endpoints to cloud infrastructure, where valuable data resides. O'Donnell introduces the concept of Canary tokens – small, digital tripwires strategically planted within M365 to alert defenders the moment an attacker interacts with them. This research highlights the efficacy of repurposing native M365 telemetry to establish a robust, agentless detection mechanism, providing invaluable early warning against both external threats and insider risks.

Watch on YouTube

Visual summary for From Lure to Alert: Implementing Canary Tokens in M365 by Ryan O'Donnell
Visual summary for From Lure to Alert: Implementing Canary Tokens in M365 by Ryan O'Donnell

Key moments

  1. 3:10 Canary tokens detect post-compromise activity in M365
  2. 8:00 Using hidden mail folders for lure placement
  3. 13:00 Four-step process to deploy Canary tokens
  4. 18:00 First alert: uncovering an insider threat
  5. 23:00 Tuning mistake led to missing an actual breach
  6. 26:00 Key lessons: collection, tuning, and native telemetry
  7. 37:20 Mail Items Accessed for linkable identifiers

From Lure to Alert: Implementing Canary Tokens in M365

Speakers: Ryan O'Donnell, Senior Security Engineer, Microsoft

Conference: Wild West Hackin' Fest @ Mile High

YouTube: https://www.youtube.com/watch?v=XXf7lIzYyUw

Overview

In his Wild West Hackin' Fest @ Mile High talk, "From Lure to Alert: Implementing Canary Tokens in M365," Ryan O'Donnell, a Senior Security Engineer at Microsoft, delves into an innovative approach for early detection of post-compromise activity within Microsoft 365 environments. The presentation addresses the critical shift in attacker focus from traditional endpoints to cloud infrastructure, where valuable data resides. O'Donnell introduces the concept of Canary tokens – small, digital tripwires strategically planted within M365 to alert defenders the moment an attacker interacts with them. This research highlights the efficacy of repurposing native M365 telemetry to establish a robust, agentless detection mechanism, providing invaluable early warning against both external threats and insider risks.

The talk underscores the escalating threat of cloud intrusions, evidenced by a nearly 140% rise observed by CrowdStrike, and the alarming dwell times reported by Mandiant, where the median time between compromise and detection can be as high as 11 days, extending to 16 days for externally detected breaches. O'Donnell’s work aims to drastically reduce these dwell times by focusing on the initial post-compromise phase, specifically when attackers begin keyword searching for sensitive information within compromised mailboxes. By demonstrating the successful deployment and lessons learned from two real-world trials, this talk provides practical, actionable insights for organizations seeking to bolster their cloud security posture without relying on expensive, complex deception platforms.

Background

▶ Watch: Canary tokens detect post-compromise activity in M365 (3:10)

The landscape of cyberattacks has fundamentally shifted, with adversaries increasingly bypassing traditional endpoint defenses to target the cloud directly. As O'Donnell emphasizes, quoting Florian Roth, "attackers are bypassing the endpoints entirely and staying purely up in the cloud using tokens to achieve their objectives." This migration is driven by the simple fact that the most valuable data – emails, invoices, credentials, intellectual property – now predominantly resides in cloud productivity suites like M365. Despite this clear trend, many organizations continue to struggle with timely detection of cloud intrusions.

A significant challenge highlighted by O'Donnell is the prolonged dwell time – the period an attacker remains undetected within an environment. Mandiant's data reveals a median dwell time of 11 days overall, with a stark 16-day gap when detection originates from external sources (third parties, law enforcement) compared to internal teams. This extended window grants attackers ample time to achieve their objectives, underscoring the urgent need for earlier detection capabilities. The core problem O'Donnell addresses is the lack of effective early warning systems tailored for cloud environments, particularly for post-compromise activities that occur after initial access has been gained, such as data enumeration and collection.

Attackers, once inside, often engage in predictable behaviors. A prime example is the advanced persistent threat (APT) group Storm 2372, known for device code phishing to steal tokens and gain initial access. Their subsequent actions frequently involve keyword searching through compromised mailboxes for terms like "username," "password," "admin," or other objective-specific phrases. This unsophisticated yet highly effective search-export-exfiltration (search-export-Xfill) pattern is a common thread across many cloud intrusions. O'Donnell's research builds upon this understanding, aiming to create "tripwires" that trigger alerts precisely when these initial search behaviors manifest, thereby moving detection much earlier in the attacker's kill chain and significantly reducing potential dwell time.

Key Findings

▶ Watch: Four-step process to deploy Canary tokens (13:00)

O'Donnell's research yielded several crucial findings through the design, deployment, and analysis of Canary tokens in two live M365 environments:

  1. Canaries Catch Collection Behavior, Regardless of Attacker Type: The most significant and perhaps unexpected finding was that Canary tokens effectively detect data collection behavior, whether initiated by an external attacker or an insider. The first successful trial, initially designed to catch external threats, instead flagged an insider risk scenario involving a departing employee collecting sensitive company information. This demonstrated the versatility of the approach: the "tripwire doesn't care who's searching, just cares that someone is doing it at a volume or looking for things that they shouldn't be." This broad applicability extends the utility of Canary tokens beyond traditional external threat hunting to critical areas like insider risk management.
  1. Tuning is a Bet on Attacker Behavior: The second pilot trial, which resulted in a "miss," underscored a critical lesson: every tuning decision and keyword exclusion is an assumption about how an attacker will behave. In this instance, necessary tuning to reduce false positives from legitimate finance team activity led to the exclusion of "invoice" and "wire" as Canary terms. Subsequently, a real external breach occurred where the attackers performed only two searches, specifically for "invoice" and "wire." This highlighted that attackers are often highly goal-oriented, not exploratory like red teamers, and may use a minimal set of highly targeted terms. Over-tuning or making broad assumptions about attacker thoroughness can lead to critical detection gaps.
  1. Native Cloud Telemetry is a Powerful Deception Platform: O'Donnell's work demonstrates that organizations do not necessarily need a full-fledged, expensive deception platform to implement effective Canary tokens. By creatively leveraging existing, native Microsoft 365 telemetry – particularly the Unified Audit Log and operations like MailItemsAccessed – defenders can build robust detection mechanisms. This approach utilizes data already available and often retained, transforming high-volume, seemingly benign events into critical security signals when correlated with specific Canary interactions. This democratizes deception capabilities, making them accessible to a broader range of organizations.
  1. Early Detection Reduces Dwell Time: The overarching goal and a key finding is the potential of these M365 Canaries to significantly move detection earlier in the attacker workflow. By focusing on the initial search and enumeration phase, organizations can gain an alert much sooner than through traditional methods, thereby reducing the critical window attackers have to achieve their objectives and potentially exfiltrate data.

Technical Deep Dive

▶ Watch: First alert: uncovering an insider threat (18:00)

O'Donnell's methodology for implementing M365 Canary tokens is a clever fusion of native M365 features, the Graph API, and intelligent telemetry analysis. The core idea is to create lures that are enticing to attackers but invisible to regular users, and then to monitor their interaction with these lures using existing audit logs.

The process begins with lure generation. O'Donnell categorizes lures into themes like "creds," "finance," and "HR," with associated keywords (e.g., "username," "password," "admin" for creds; "invoice," "wire transfer" for finance). Crucially, each lure needs not just keywords but also a "cover story" or context to appear legitimate. The challenge lies in making these lures discoverable by an attacker but not by a legitimate user, to minimize false positives.

This is where hidden mail folders come into play, a technique O'Donnell discovered through a blog from Sotaria. The Graph API supports a property called is_hidden on mail folders. When set to true, a folder will not appear in Outlook (desktop or web app, OWA) or on mobile devices. This provides an excellent clandestine location for lure emails. A practical note: it takes about 10-15 minutes for a folder to visibly disappear, so deployment during off-hours is recommended.

Despite being hidden from the UI, emails within these hidden folders remain searchable. O'Donnell demonstrated this using Graphrunner, a Graph API post-exploitation toolkit, showing that keyword searches successfully returned results from emails residing in a hidden folder. This ensures that an attacker performing enumeration will still discover the lures, while a casual user browsing their inbox will not stumble upon them.

The detection mechanism relies on the Unified Audit Log within M365, specifically the MailItemsAccessed operation. This event registers whenever an email is accessed, opened, or read. While MailItemsAccessed is a high-volume event, O'Donnell leverages the Internet Message ID – a unique identifier for each email – to specifically monitor interactions with the Canary lure emails. A simple KQL query can be used to filter for this specific ID.

The telemetry for MailItemsAccessed is accessible through various channels:

  • XDR and advanced tables
  • Cloud App Events
  • Office 365 Management API
  • PowerShell (via Exchange Online Management cmdlets)
  • Graph API

A critical prerequisite is that mailbox auditing must be enabled. While this is typically on by default for E3 and E5 licenses, small business licenses may require manual activation.

Upon an alert (i.e., the MailItemsAccessed event for a Canary email), rapid investigation is key. O'Donnell highlights the importance of linkable identifiers such as the Session ID and Unique Token ID found within the audit record. These identifiers allow defenders to immediately pivot and correlate other activities associated with that specific session or access token. This "three-step process" – pulling the Session ID, unrolling other events, and correlating with other information sources – is crucial for quickly distinguishing between legitimate access and malicious activity, especially given the high volume of MailItemsAccessed events.

The overall technical process can be summarized as:

  1. Create a hidden folder using the Graph API.
  2. Plant a lure email (with relevant keywords and context) inside the hidden folder, meticulously tracking its Internet Message ID.
  3. Consistently monitor the Unified Audit Log for MailItemsAccessed events specifically tied to the Canary's Internet Message ID.
  4. Upon alert, immediately use the Session ID and Unique Token ID to pivot and investigate all other activities performed by that token or session to determine if it's an attacker.

It's important to note the scope and limitations: this method targets the common post-compromise search-export-Xfill workflow. It won't catch attackers who never search for the planted keywords. Also, the focus is on bind events (individual item access) rather than sync events (folder synchronization), although O'Donnell notes later adaptations to detect hidden folder syncs as well.

Demo / Proof of Concept

▶ Watch: Key lessons: collection, tuning, and native telemetry (26:00)

O'Donnell validated his Canary token methodology through two distinct production pilots in small company environments, each offering valuable lessons.

Pilot 1: The Insider Threat "Win"

The first trial involved deploying approximately 50 Outlook Canaries across various mailboxes over an initial three-month duration. Out of three alerts, one proved to be a significant "win," albeit for an unexpected scenario: insider risk.

The alert triggered at 11:47 PM on Day 12 of deployment. O'Donnell immediately initiated an investigation, scrutinizing IP addresses, locations, devices, applications, and suspicious operations like new inbox rules, hard deletes, or file movements. Initially, nothing definitively malicious stood out, leading him to consider it a false positive. However, a "tingly feeling" prompted him to reach out to the company's point of contact (PC).

The PC provided crucial context: the employee whose Canary had triggered was notified of their impending termination just two days prior to the alert. This reframed the investigation. O'Donnell then focused on the volume of email accesses by that user. Analysis revealed a significant spike, more than triple the usual activity, during the week the user was notified and the Canary triggered. Further information, later confirmed, revealed the user had accepted a job with a competitor and was likely attempting to collect customer information valuable to their new role.

This incident entirely reshaped O'Donnell's perception of Canary tokens. While built with external attackers in mind, the first win was an insider risk scenario with a legitimate user. The key insight: the Canary fired because the collection behavior was the same, regardless of the actor. This validated the core premise that Canaries detect suspicious enumeration, whether by an external threat or a malicious insider. O'Donnell reinforced this by citing a recent news story about a former Google employee found guilty of economic espionage for siphoning AI-related files, a collection activity that Google reportedly discovered by chance, not detection.

Pilot 2: The Tuning "Miss"

The second pilot used the same setup but in a different organizational environment. Canaries were deployed with standard contexts (creds, finance, HR). However, this environment presented a unique challenge: the finance team heavily relied on their inbox, constantly searching for terms like "invoice" and "wire transfer" as part of their legitimate job duties. This generated a high volume of false positives.

Faced with the choice to "drown in false positives" or tune them out, O'Donnell and his team opted for tuning, specifically excluding "invoice" and "wire" from the Canary terms, while retaining "payroll" and "payment." This decision was based on the assumption that an actual attacker would likely search more broadly than just two terms.

This assumption proved incorrect. Just weeks after the tuning, a real external breach occurred due to compromised credentials and unauthorized access – precisely the scenario the Canaries were designed to catch. However, no alerts fired. Upon reviewing the logs, it was discovered that the attackers performed only two searches, specifically for "invoice" and "wire." Had those terms not been tuned out, the Canaries would have fired.

This "miss" provided a crucial second lesson: tuning decisions are assumptions about attacker behavior, and these assumptions can be wrong. O'Donnell, drawing on his experience as a pentester and red teamer, realized he had projected a red teamer's thoroughness onto the attackers. In reality, business compromise attackers are often highly focused and efficient, going "straight for" a "very specific singular goal" without extensive exploration. The fix involved rebuilding the detection with more environment-specific baselines and incorporating a baseline period before implementing blanket exclusions.

Defensive Implications

▶ Watch: Mail Items Accessed for linkable identifiers (37:20)

The implementation of Canary tokens in M365, as demonstrated by Ryan O'Donnell, offers several critical defensive implications for organizations:

  • Proactive, Early Detection: Defenders should consider deploying M365 Canary tokens to gain early visibility into post-compromise activity. This method focuses on the attacker's initial enumeration and collection phase, providing alerts significantly earlier than many traditional detection mechanisms and drastically reducing potential dwell time.
  • Leverage Native Telemetry: Organizations do not need to invest in complex, external deception platforms. By intelligently repurposing existing Unified Audit Log data, particularly MailItemsAccessed events and associated linkable identifiers, defenders can build effective, agentless detection systems directly within their M365 tenant.
  • Holistic Threat Coverage (External & Insider): Canaries prove effective against both external attackers and insider threats. This dual capability makes them a valuable addition to an organization's security stack for comprehensive risk management, especially during sensitive periods like employee offboarding.
  • Contextual Lure Design: The success of Canary tokens hinges on well-crafted lures. Defenders should design lures with compelling keywords and plausible cover stories, tailored to their organization's specific data, operational context, and potential attacker objectives.
  • Strategic Placement with Hidden Folders: Utilizing the Graph API to create is_hidden mail folders is a powerful technique for clandestine lure placement. This ensures lures are discoverable by programmatic searches (used by attackers) but invisible to casual user browsing, minimizing accidental triggers.
  • Intelligent Tuning and Baselining: Avoid blanket exclusions or tuning based solely on generic threat intelligence or red team assumptions. Instead, implement a baselining period to understand legitimate user behavior within the specific environment. Tuning should be iterative, data-driven, and environment-specific, acknowledging that attackers can be highly targeted.
  • Robust Investigation Workflows: When a Canary fires, a rapid and structured investigation is crucial. Defenders should leverage Session ID and Unique Token ID from audit logs to quickly pivot and correlate other activities, enabling efficient triage of false positives and confirmation of malicious intent.
  • Ensure Mailbox Auditing is Enabled: Verify that mailbox auditing is universally enabled across the M365 tenant, especially for small business licenses where it might not be the default. This is foundational for capturing the necessary MailItemsAccessed telemetry.
  • Consider "Meta-Canaries" and Role-Based Context: For high-volume terms, O'Donnell suggests exploring "meta-canaries" – linking an initial Canary trigger to a subsequent, more active step (e.g., if a finance lure is accessed, check if the attacker then tries to interact with a linked email address). Additionally, mapping job roles to expected search behaviors can help refine detection logic and reduce false positives. Incorporating baselining around user IP address history and device IDs can further enhance detection accuracy.
  • Expand Detection to Sync Events: While initially focused on bind events, defenders should also consider creating detections for sync events of hidden Canary folders, as attackers might sync entire folders offline.

Key Takeaways

  • Canary tokens in M365 are effective tripwires for detecting data collection behavior, whether by external attackers or malicious insiders, highlighting the importance of early detection regardless of the adversary's origin.
  • Tuning detection rules, especially keyword exclusions, is a calculated risk based on assumptions about attacker behavior; environment-specific baselining is crucial to avoid missing targeted attacks.
  • Organizations can build powerful deception capabilities using native M365 telemetry, such as the Unified Audit Log's MailItemsAccessed events, without requiring dedicated deception platforms.
  • Strategic placement in hidden mail folders (via Graph API's is_hidden property) ensures lures are discoverable by attackers but invisible to legitimate users, minimizing false positives.
  • Leveraging linkable identifiers like Session ID and Unique Token ID within audit logs is vital for rapidly pivoting and investigating an alert to determine its veracity and scope.
  • Canary tokens significantly contribute to reducing attacker dwell time by catching enumeration and collection activities at an early stage of the post-compromise workflow.

About the Speaker(s)

Ryan O'Donnell is a Senior Security Engineer at Microsoft. With a background heavily rooted in the offensive side of cybersecurity, he has transitioned his expertise to developing robust detection mechanisms, specifically for behaviors he once exploited. His research and talks, like this one, are presented as his personal work and do not represent the official stance or confidential data of his employer, Microsoft, focusing instead on publicly accessible information and techniques applicable to customer-facing environments.

Reviews

Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT

This is how defensive engineering should look. O'Donnell takes a basic concept—canary tokens—and wires it natively into M365 using the Graph API and Unified Audit Log, skipping the vendor tax entirely. It's highly actionable, bullshit-free, and grounded in actual production deployments.

Heather Calloway (CISO) — STRONG ACCEPT

This is an exceptionally practical blueprint for cutting cloud dwell times without buying another deception vendor. O'Donnell proves that native M365 telemetry, when paired with simple hidden folders, creates high-fidelity tripwires for both external attackers and malicious insiders. It’s a low-cost, high-yield operational win that your security engineering team should be testing this quarter.

→ Top-rated talks at Wild West Hackin' Fest @ Mile High 2026

All talks from Wild West Hackin' Fest @ Mile High 2026