Catching Collection in M365: Outlook and SharePoint Canary Tokens
Ryan O'Donnell (Senior Security Engineer · Microsoft)
RVAsec 2026 · Day 2 · Grand Ballroom F/G
Overview
In an era where cloud intrusions are rapidly escalating, traditional endpoint-centric defenses often fall short. This talk by Ryan O'Donnell, a Senior Security Engineer at Microsoft, addresses the critical challenge of detecting post-compromise data collection within Microsoft 365 environments. O'Donnell introduces a practical and accessible approach to deploying Canary tokens within Outlook and SharePoint, designed to provide early warning signals when attackers, or even malicious insiders, begin searching for valuable data. The core premise is to leverage native M365 telemetry to identify suspicious access to these decoy assets, offering a high-fidelity tripwire in the crucial "collection" phase of an attack.

Key moments
- 0:00 Introduction and first real-world alert
- 3:46 Canary tokens for M365 post-compromise early warning
- 7:00 Access-based detection using native Unified Audit Log
- 15:00 Deploying Outlook Canaries in hidden folders
- 23:40 Pilot 1: Detecting insider risk with Outlook Canaries
- 37:00 Pilot 2: Lessons from intrusion and tuning assumptions
- 41:50 Key takeaways: Collection, tuning, and UAL value
Catching Collection in M365: Outlook and SharePoint Canary Tokens
Speakers: Ryan O'Donnell, Senior Security Engineer, Microsoft
Conference: RVAsec
YouTube: https://www.youtube.com/watch?v=EWvleH0zWH8
Overview
In an era where cloud intrusions are rapidly escalating, traditional endpoint-centric defenses often fall short. This talk by Ryan O'Donnell, a Senior Security Engineer at Microsoft, addresses the critical challenge of detecting post-compromise data collection within Microsoft 365 environments. O'Donnell introduces a practical and accessible approach to deploying Canary tokens within Outlook and SharePoint, designed to provide early warning signals when attackers, or even malicious insiders, begin searching for valuable data. The core premise is to leverage native M365 telemetry to identify suspicious access to these decoy assets, offering a high-fidelity tripwire in the crucial "collection" phase of an attack.
O'Donnell, leveraging his background in offensive security, details the design, implementation, and real-world efficacy of these M365-specific canaries. He shares insights from two production pilots conducted in small business environments, revealing not only the success of detecting an insider threat but also the pitfalls of making assumptions about attacker behavior during tuning. The presentation emphasizes an access-based detection model, moving beyond link-based triggers to alert on the mere interaction with a canary asset, thereby simplifying adoption for organizations that might lack extensive security tooling.
Background
▶ Watch: Introduction and first real-world alert (0:00)
The landscape of cyber intrusions has shifted dramatically, with attackers increasingly bypassing traditional endpoint defenses to operate directly within cloud environments. As highlighted by a Florian Roth quote, "attackers kind of bypassing the endpoints entirely and operating purely in the cloud. They don't have any shells or malware, just tokens interacting with the Microsoft Graph API." This trend is empirically supported by reports such as CrowdStrike's 2025 report, which indicated a staggering 140% increase in cloud intrusions year-over-year. The motivation is clear: data resides in the cloud, and attackers follow the data. For a vast number of organizations, this critical data lives within Microsoft 365 (M365) services like Outlook, SharePoint, and Teams.
Once initial access is gained, often through token theft or phishing, a common and critical next step for adversaries is data collection. Microsoft Threat Intelligence Center (MSTIC) reports, for instance, detail how threat actors like Storm-2372, after stealing tokens, immediately search compromised mailboxes for keywords such as "username," "password," and "admin." Similarly, Google's threat intelligence group has observed actors like BlackFile scouring SharePoint and OneDrive for terms like "confidential" and "social security number." This consistent pattern of data-centric post-compromise activity presents a prime opportunity for early detection. While deception technologies have existed for a long time, O'Donnell's research focuses on creating an easier, simpler way for small businesses to adopt Canary tokens that trigger alerts based on resource access itself, rather than requiring an additional step like clicking an embedded link. This approach leverages native M365 telemetry, specifically the Unified Audit Log (UAL), to minimize the need for additional platforms or subscriptions, making it more accessible for organizations with limited security budgets or resources.
Key Findings
▶ Watch: Access-based detection using native Unified Audit Log (7:00)
The research and production pilots unveiled several critical findings regarding the deployment and efficacy of Canary tokens in M365:
- Effective Early Warning: Canary tokens, when strategically placed in Outlook and SharePoint, provide a high-fidelity, early warning signal for post-compromise data collection activities by both external attackers and insider threats. This addresses a crucial gap in many organizations' detection capabilities.
- Broad Applicability of Collection Detection: The initial "win" from the first pilot demonstrated that Canary tokens are not limited to detecting external adversaries. The detection of an offboarding employee engaging in significant data collection highlighted their utility in identifying insider risk, reframing the potential scope of these deception mechanisms.
- Outlook Hidden Folders for Low Noise: The discovery and utilization of the
is_hiddenproperty in the Microsoft Graph API for Outlook folders proved instrumental. This allows Canary emails to be placed in folders that are invisible during normal Outlook navigation but remain fully searchable by automated tools or manual keyword searches, drastically reducing false positives from legitimate user interaction. - Tuning is a Bet: The second pilot starkly illustrated that tuning detection rules based on assumptions about attacker behavior is a significant risk. Removing canaries with "invoice" or "wire transfer" context, despite high legitimate user activity, led to a missed detection during an actual business email compromise (BEC) where the attackers only searched for those exact terms.
- Importance of Baselining: To mitigate the "tuning is a bet" problem, O'Donnell emphasized the necessity of a stringent baselining period (e.g., 14 days to a month) in new environments. This allows defenders to understand actual user behavior and activity patterns before making informed decisions about which terms or locations might generate excessive noise.
- Unified Audit Log as a Rich Source: The Unified Audit Log (UAL) in M365 was confirmed as an incredibly rich and native source of telemetry. Despite its potential messiness and high volume, it contains sufficient data—including crucial correlation keys and linkable identifiers—to build effective, high-fidelity detections without requiring additional third-party tools or complex ingestion pipelines.
Technical Deep Dive
▶ Watch: Deploying Outlook Canaries in hidden folders (15:00)
The core of O'Donnell's approach lies in leveraging native M365 telemetry to achieve access-based detection. The goal is to alert when a canary item is accessed during an attacker's search-driven data collection phase, not just when a link is clicked. This is achieved through a systematic monitoring and investigation workflow:
- Monitor UAL Events: Continuously monitor specific operations in the Unified Audit Log (UAL) for activity related to canary files.
- Pivot on Alert: Upon an alert, immediately pivot to identify the
UserID,ClientIP,ApplicationID, and other contextual details of the access. - Scope Investigation: Expand the investigation to determine "what else happened in the same session?" by leveraging linkable identifiers to correlate activities back to an authentication session or a unique token ID.
The Unified Audit Log (UAL)
The UAL is the centralized data source for all Microsoft 365 audit events, encompassing activities from Teams, SharePoint, OneDrive, and Exchange Online. While it can be voluminous, it's native to M365 and can be ingested via the Office 365 Management Activity API or PowerShell, allowing organizations to feed it into a SIEM like Microsoft Sentinel or a custom database. Crucially, O'Donnell highlights tools like the Invectus Microsoft Extractor Suite for UAL ingestion.
For canary detection, specific UAL events are paramount:
- Outlook Canaries: The
MailItemsAccessedevent, specifically theBindoperation. This event is high-volume, generated whenever users interact with emails, but contains critical details likeUserID,AppID, and theInternetMessageId. - SharePoint Canaries: The
FileDownloadedandFileAccessedevents. These provide details similar toMailItemsAccessed, includingUserID,AppID, and the crucialListItemUniqueId.
Correlation and Investigation
To distinguish legitimate access from malicious activity, O'Donnell emphasizes the use of linkable identifiers within the UAL. These include UniqueTokenIdentifier and CorrelationID, which allow security teams to trace activity back to an authentication session. As demonstrated by Matt Graber at Forward Cloudstack, this enables correlating an event (e.g., a New-InboxRule creation) back to the non-interactive and then interactive sign-in events, providing a comprehensive view of the attacker's initial access and subsequent actions.
Outlook Canary Implementation
- Canary Creation: Canaries are crafted with threat-informed keywords and plausible cover stories. Themes include "VPN Credentials," "HR Reports," and "Wire Transfer Details," designed to mimic valuable data attackers would seek (e.g., "username," "password," "admin," "anydesk").
- Hidden Folder Placement: To minimize false positives from legitimate user interaction, O'Donnell leverages a lesser-known feature of the Microsoft Graph API: the
is_hiddenproperty for folders. Settingis_hiddentotruemakes a folder invisible in normal Outlook navigation. Critically, messages within these hidden folders remain searchable, allowing an attacker using tools like Graphrunner (a post-exploitation tool that searches mailboxes) to discover the canary email. A minor gotcha: the hiding isn't instantaneous, taking about 15 minutes or a client refresh. - Monitoring: The unique identifier for an Outlook canary email is its InternetMessageId. When deploying a lure email, this ID must be recorded. Monitoring involves a simple KQL query (or similar logic in a custom system) that checks for
MailItemsAccessedevents where theInternetMessageIdmatches that of the canary. - Prerequisites: Unified Audit Log must be enabled (though often on by default in newer tenants, older ones might need manual activation), and Mailbox Auditing must be turned on.
SharePoint Canary Implementation
- Canary Creation: Similar themes are used for SharePoint files, but with more flexibility for content, file names, folder paths, and metadata to enhance plausibility.
- Placement: Unlike Outlook, there isn't a direct "hidden folder" equivalent for SharePoint files that remains searchable but invisible to regular browsing. Therefore, canaries must be placed in plausible, public locations within SharePoint sites, such as finance or HR folders, where attackers would naturally search.
- Monitoring: The correlation key for SharePoint files is the ListItemUniqueId. This durable identifier is used to monitor for
FileDownloadedorFileAccessedevents. A custom monitoring solution, exporting data from the Management Activity API, can check for these events against the recordedListItemUniqueIdof the canaries. - Distinguishing Access: Beyond the correlation keys, additional fields in the UAL events, such as
UserAgentandClientAppID, are crucial. O'Donnell references a Google threat intelligence report that identified mass data exfiltration via a user agent and client app ID mismatch (e.g., a Python script masquerading with a legitimateClientAppID), indicating suspicious automated access. - Prerequisites: The Unified Audit Log must be configured to retain relevant events and track the
ListItemUniqueId.FilePreviewedevents were deemed too noisy and unreliable for this purpose.
Demo / Proof of Concept
▶ Watch: Pilot 2: Lessons from intrusion and tuning assumptions (37:00)
Instead of a live demonstration, the talk presented the results of two real-world "production pilots" conducted in small business environments. These pilots served as crucial proof-of-concept validations for the M365 Canary token strategy.
Production Pilot 1: Outlook Canaries and Insider Risk
The first pilot involved deploying 50 Outlook canaries in a small business environment over a three-month period. Three alerts were generated, two of which were benign. The most significant alert, occurring on Day 12, initially appeared unusual but not overtly malicious. The investigation involved scrutinizing IP addresses, locations, device statuses, and applications used to access the canary.
The breakthrough came from communicating with the organization's point of contact. It was revealed that the user who triggered the alert had recently been notified of their upcoming offboarding. Further analysis of mailbox activity volume around this period showed a tripling of activity in Week 4, coinciding with the offboarding notification. While initially thought to be routine cleanup, additional context confirmed the user was preparing to join a company that would benefit from their current employer's customer data. This reframed the detection: the canary had caught significant collection-type behavior, not from an external attacker, but from an insider threat. O'Donnell drew parallels to the recent AI economic espionage conviction of a Google engineer who exfiltrated 2,000 pages of AI trade secrets, emphasizing that while the threat actor differs, the underlying collection behavior remains the same. The canary served as the "tripwire" for this behavior.
Production Pilot 2: Outlook and SharePoint Canaries, and the Peril of Tuning
The second pilot expanded the scope to include both Outlook and 10 SharePoint canaries across four different sites/folders, maintaining similar themes (finance, HR, credentials). This environment presented a unique challenge: the finance team constantly searched for templates and reports containing terms like "invoice" and "wire transfer." To reduce expected noise, a decision was made to remove SharePoint canaries with "invoice" or "wire transfer" context and tune out similar Outlook canaries, while retaining other financial terms like "payroll" and "payment."
Unfortunately, this tuning decision proved costly. A few months into the pilot, an actual intrusion occurred: an external attacker compromised credentials, accessing mailboxes and SharePoint. The initial alert was for a classic business email compromise (BEC) attempt targeting the finance department. However, no canary alerts were triggered. A post-incident review revealed that the attackers performed only two searches in both Outlook and SharePoint: "invoice" and "wire." Because these specific terms had been tuned out, the canaries remained silent. This highlighted a crucial lesson: tuning is a bet on attacker behavior. Attackers, especially those focused on specific objectives like BEC, often don't explore broadly; they go directly for what they need. This led to a revised strategy: implement a more stringent baselining period (14 days to a month) in new environments to understand actual user search patterns before making tuning decisions, ensuring that critical terms are not prematurely filtered out.
Defensive Implications
▶ Watch: Key takeaways: Collection, tuning, and UAL value (41:50)
Organizations can significantly enhance their defensive posture against post-compromise data collection by implementing the M365 Canary token strategy outlined by O'Donnell. Key defensive implications include:
- Proactive Collection Detection: Deploy Canary tokens in M365 (Outlook and SharePoint) to establish early warning systems for data collection, a common and critical phase in almost every cloud intrusion. This shifts detection left in the attack chain.
- Leverage Native Telemetry: Maximize the use of the Microsoft 365 Unified Audit Log (UAL). It's a rich, native data source that can provide high-fidelity alerts without requiring additional security products or subscriptions. Ensure UAL and Mailbox Auditing are enabled across all tenants.
- Target Specific Events: Focus monitoring efforts on
MailItemsAccessed(specificallyBindevents) for Outlook andFileDownloadedorFileAccessedfor SharePoint. These events provide the necessary granularity for access-based detection. - Utilize Durable Identifiers: For reliable canary monitoring, use the InternetMessageId for Outlook emails and the ListItemUniqueId for SharePoint files. These unique identifiers are stable and prevent false negatives from changes in subject lines, file names, or folder paths.
- Enhance Investigation with Linkable Identifiers: Integrate the use of
UniqueTokenIdentifierandCorrelationIDfrom the UAL and sign-in logs into incident response playbooks. These identifiers enable comprehensive session-based investigations, correlating canary alerts back to initial authentication events and other related activities. - Implement Low-Noise Outlook Canaries: Leverage the
is_hiddenproperty in the Microsoft Graph API when creating Outlook folders. This allows canaries to be hidden from normal user view while remaining discoverable by attacker search tools, significantly reducing benign positives. - Strategic SharePoint Canary Placement: For SharePoint, carefully place canaries in plausible, public locations (e.g., finance, HR shared drives) where attackers would naturally look for sensitive data, accepting the trade-off that they might be visible to regular users.
- Rethink Tuning with Baselines: Avoid making assumptions about attacker search patterns when tuning. Instead, implement a stringent baselining period (e.g., 14-30 days) to understand normal user behavior and search terms within an environment before filtering out potential canary terms. This mitigates the risk of missing critical alerts, as demonstrated by the BEC incident.
- Broaden Threat Scope: Recognize that Canary tokens can detect not only external adversaries but also insider threats and potentially even malicious autonomous agents (e.g., AI copilots). The detection mechanism is agnostic to the actor, focusing purely on the collection behavior.
- Monitor for Anomalous Access Patterns: Beyond canary hits, defenders should look for suspicious patterns in UAL data, such as user agent/client app ID mismatches (e.g., a Python script accessing M365 resources) or unusual spikes in data access volume, as these can indicate malicious activity.
Key Takeaways
- Canary tokens in M365 provide effective early warning for post-compromise data collection by external attackers, insider threats, and even potentially AI-driven agents.
- Tuning detection rules based on assumptions about attacker behavior is a significant risk; thorough baselining of user activity is crucial to avoid missing critical alerts.
- The Microsoft 365 Unified Audit Log (UAL) is a powerful, native, and often underutilized telemetry source for building high-fidelity detections without requiring additional security tools.
- Access-based detection, leveraging specific UAL events like
MailItemsAccessed(Bind) andFileDownloaded/FileAccessed, offers a robust method for identifying suspicious interaction with canary assets. - Utilizing the
is_hiddenproperty for Outlook folders via the Microsoft Graph API allows for the deployment of highly effective, low-noise email canaries that remain searchable to attackers. - Reliable monitoring hinges on using durable identifiers like
InternetMessageIdfor Outlook emails andListItemUniqueIdfor SharePoint files.
About the Speaker(s)
Ryan O'Donnell is a Senior Security Engineer at Microsoft. With a background rooted in offensive security, including pen testing and red teaming, O'Donnell brings a unique perspective to defensive strategies. His current focus lies in deception technologies, aiming to identify and counter the very behaviors he once exploited. The research presented at RVAsec, including the conducted production pilots, was part of his personal endeavors, tested in small business environments outside of his regular day job responsibilities.
Reviews
Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT
Ryan O'Donnell takes a tired concept—canary tokens—and actually makes it work natively inside M365 without buying another appliance. Abusing the Graph API to build invisible but searchable honeypots is exactly the kind of clever, zero-budget defense that moves the needle.
Heather Calloway (CISO) — STRONG ACCEPT
A highly practical guide to building deception tripwires using native M365 telemetry instead of buying another vendor platform. The speaker's honest post-mortem on how tuning for alert fatigue caused them to miss a live BEC intrusion is required reading for any detection team.