macOS Vulnerability Research: Augmenting Apple's Source Code and OS Logs with AI Agents

Diff to Hypothesis to Harness: How Olivia Gallucci Uses AI Agents for macOS Vulnerability Research

Speaker: Olivia Gallucci, Security Engineer, Datadog

Conference: [un]prompted 2026 — The AI Security Practitioner Conference

Date: March 4, 2026, San Francisco

Watch on YouTube: https://www.youtube.com/watch?v=_f30RyXc_8Q

Reading time: ~9 minutes

TL;DR: Olivia Gallucci built a workflow that treats Apple's partial open-source code as a sensor stream — using AI agents to turn source diffs, proprietary binaries, and OS logs into a prioritized vulnerability research queue. The model is not God; it's a junior analyst that reads fast, follows procedures, and produces structured hypotheses for humans to validate.

A Tool Named OSS Sensor — and Why There Are Two Versions

▶ Watch: Introduction and OSS Sensor overview (0:00)

Gallucci opened by introducing a tool she published called OSS Sensor (available at github.com/oliviagallucci/oss-sensor). OSS Sensor generates a prioritized, evidence-backed research queue derived from Apple's open-source diffs, basic binary features, and log templates. It applies custom rule-based scoring and supports rule-based triage. Critically: the binary and log analysis components are intentionally lightweight, and there is no integrated LLM vulnerability verification built in.

"It's designed as a triage and hypothesis generation aid, not an automated vulnerability discovery tool."

There are two versions. The non-AI version is more streamlined and significantly easier to adapt — Gallucci recommends starting there, especially for anyone planning to integrate their own AI workflow. This way the tool is personalized to your setup, not dependent on her configuration choices. The plug-and-play version with AI API keys is still under PR review, because her workflow includes intentional manual intervention steps that she's not ready to automate away.

Her background gives her perspective on why this matters operationally: she previously worked at Apple, SECUINFRA GmbH, and the U.S. government before joining Datadog. A year ago, she noted, many AI chatbots couldn't even tell you how to access logs on macOS.

The Core Thesis: Apple's Open Source as a Sensor

▶ Watch: The sensor model and research scope (2:00)

The central insight driving Gallucci's workflow is treating Apple's partial open-source code not as an incomplete documentation source, but as a sensor — a data stream that produces signals about system state changes without doing the interpretation for you.

The research narrative she built around this is: diff to hypothesis to harness.

1. Start with a diff (source or binary)
2. Ask what meaningfully changed and where trust boundaries might be
3. Turn that into a testable hypothesis about a bug class
4. Build a minimal test harness so the path is reachable and reproducible
5. Feed into fuzzing and validation

Apple has over 400 public repositories. The releases are incomplete and very hard to keep up with manually. Some components are missing or only partially available — including Core Foundation, CFNetwork, iOS-specific drivers, and CoreCrypto (which comes with additional redistribution constraints). Headers reference files that don't exist; instructions are missing. "Which can be very confusing for an AI agent."

The augmentation trick is to combine three signals that Apple's open source alone can't provide: source diffs (what changed), binaries (what actually shipped), and OS logs (what actually ran).

Why Information Overload Is the Problem AI Solves

▶ Watch: Why AI agents help here (4:00)

Without automation, a researcher doing manual repo exploration and one-off diffs gets a narrow view of risk. With automation, the bookkeeping is handled, freeing time for the parts where AI still underperforms: reasoning about privilege boundaries and exploitability.

"If I rely on a human — AKA myself — to do manual repo exploration, I end up with a narrow view of risk. If I automate that bookkeeping, I can spend more time on the parts that AI isn't working well for me yet."

Gallucci even found the approach useful for threat detection engineering. The faster she understands what changed in Apple's stack, the faster she can build detections and prioritize around new attack mechanics.

The key philosophical framing: treat Apple's open source like a map. "You might not have the full territory, but the landmarks are real."

The Binary Diffing Gold Standard

▶ Watch: Binary diffing — the syslogd example (6:00)

Gallucci cited her all-time favorite example of what this workflow can surface: in 2016, the researcher OSX Reverser used Diaphora to diff syslogd between macOS 10.11.2 and 10.11.3. Only one function had changed. The change: value + 4 became value * 4 + 4.

"From an exploitation mindset, that's suspicious — it looks like a heap overflow mitigation." A nearby string stated "add lockdown session, realloc failed" — a hint pointing back to the source. This kind of surgical diffing is how vulnerabilities get reversed after a patch. These fixes often need to be applied in multiple locations; if identical code is patched in some files but not others, the unpatched instances remain vulnerable.

Imagining this workflow running continuously and automatically on every Apple release — ingesting new OSS tags, tarballs, and distribution manifests, computing diffs at the function and semantic level, scoring changes for patterns like allocation math, bounds checks, entitlement gating, XPC parsing, and IOKit external method table changes — that's what Gallucci is building toward. "For my solo offensive research, this was a throughput multiplier."

She's only tested this on small individual components due to cost constraints. At organizational scale, it would compress the loop from "Apple shipped something" to "we've identified shifted behaviors and surfaces so we can adjust our focus, monitoring, and triage faster."

Architecture: Retrievers, Toolboxes, and the Agent Pipeline

▶ Watch: The agent pipeline architecture (8:00)

Gallucci's preferred architecture is an agent pipeline: a coordinated set of tool-using agents that turns raw inputs (diffs, binaries, logs) into evidence-backed outputs (ranked triage queues, hypotheses, fuzz and telemetry plans).

The retriever is the indexing layer — it searches embedded information (OSS code, symbols, notes, log templates) and returns the most relevant context for a given question or artifact.

The toolbox is a curated set of deterministic analyzers and system commands: strings, otool, class-dump, log queries, Apple distribution manifests. The agent calls these to gather facts rather than guess. Four tools show up constantly:

• strings — debug messages, format strings, file paths
• what — version and project info
• nm — symbol names in unstripped binaries and frameworks
• class-dump — interface information in private frameworks and Objective-C method surfaces

The agents handle diff triage, reverse engineering context, hypothesis generation, fuzz planning, and analysis as separate roles.

The critical design principle: "I don't treat the model as God. I treat it as a junior-junior analyst that can read fast, follow procedures, and generate structured hypotheses that we can validate with reversing, tracing, and fuzzing."

Logs as a Pivot, Not a Doctrine

▶ Watch: Unified logging as a research signal (10:00)

A key addition to classic macOS vulnerability research is Unified Logging. Gallucci treats unified logs as a pivot into attack surface and code paths — not as doctrine, because they're noisy and bulky.

What logs are good for:

• Identifying which components actually handled an input (subsystem, category, sender image info)
• Surfacing error paths that reveal parsing assumptions and type expectations
• Reconstructing data flow diagrams without having full source
• Mapping execution to attack surface when source isn't available

"Unified Logging gives you subsystem and category labels that map back to components, stable message releases, and strings that can help you hunt in binaries and source, as well as execution hints that you can use to reconstruct data flow diagrams without the full source."

This matters for detection engineering too: when the privileged AI pipeline learns a new untrusted input boundary or a new privileged XPC entry point, you can translate it into telemetry requirements and correlation rules.

Fuzz Planning: What the Agent Produces

▶ Watch: The fuzz planner agent (12:01)

The fuzz planner agent doesn't produce exploits — it produces plans. Specifically:

1. Enumerate candidate surfaces: syscall handlers, IOKit user clients, XPC services, parsers — entry points where attacker-controlled inputs actually cross trust boundaries.
2. Produce a minimal test harness: invocation path, entitlement requirements, expected input shape, ensuring the target is actually reachable and reproducible at scale.
3. Recommend a seed strategy: extracting dictionaries from strings, mining parameters from logs, bootstrapping from observed traffic so coverage ramps quickly.
4. Define success metrics: crash bucketing, sanitizer signals, unique stack traces.

"What this fuzz planner agent produces is not an exploit unless like you made it do that somehow. It's a plan."

The constraint that AI excels at here: bug class prediction and harness design. Human researchers own exploitability. This division keeps larger AI models cooperative, given that many implement mechanisms against generating full exploit chains for offensive research.

Where It Breaks Down — and Cost Reality

▶ Watch: Where the workflow breaks down and cost constraints (14:01)

Gallucci was refreshingly honest about limitations. The workflow breaks down when asked for full chain exploits. Models will occasionally invent steps, especially when given large subsystems and significant token budgets. Manual review checkpoints are necessary throughout.

Cost is the hard constraint. "When you start putting entire subsystems of operating systems into an LLM context, you hit a grand quickly." She's tested on small components only, never the full OS. Her setup — tens of thousands of dollars in research devices — prompted a self-aware observation: "I'm talking about accessibility, but is the average person gonna be able to afford this?"

At the end of Q&A, she summarized her funding requirements with characteristic directness: "Just give me lots of money." The audience approved.

Legal and compliance considerations also apply: open source releases, obfuscation, and licensing all need careful handling. The pipeline must be "compliant by design" with local indexing, careful source drop handling, and provenance tracking so every conclusion can be explained.

The Takeaway: From Sensor to Research Queue

▶ Watch: Summary and outcomes (16:01)

The core idea Gallucci left with the audience: Apple's partial open source can be treated as a sensor stream. It's not the full system, but it's a steady source of change signals, and those signals become much more valuable when processed continuously.

The workflow takes that sensor and turns it into a prioritized research queue using AI agents, with a specific philosophy: deterministic tooling does the work; the model's job is to stitch together evidence, retrieve the right context, and produce structured outputs that make a human researcher faster and more consistent.

The pipeline also shortens the loop for threat detection engineering. When you can quickly identify what changed and what behaviors moved, you can update monitoring, prioritization, and triage faster — turning platform churn into detection coverage.

She publishes a newsletter called Ret2Read focused on Apple security research, approximately monthly, for anyone who wants to follow her work.

Notable Quotes

"I don't treat the model as God. I treat it as a junior-junior analyst that can read fast, follow procedures, and generate structured hypotheses that we can validate with reversing, tracing, and fuzzing." (10:00)

"You might not have the full territory, but the landmarks are real." (on Apple's partial open source, 4:00)

"It's designed as a hypothesis-generation aid, not an automated vulnerability discovery tool." (0:00)

"Just give me lots of money." (on the cost constraints of scaling this research, 20:01)

Key Takeaways

• Apple's 400+ open source repos are an underutilized sensor stream for vulnerability research — incomplete, but steady. Agents can process that stream continuously in ways no human can match.
• The "diff to hypothesis to harness" narrative is the practical shape of AI-assisted vulnerability research: source change → security hypothesis → testable fuzz target.
• Binary diffing remains the gold standard for post-patch vulnerability identification; even a single changed line (value + 4 → value * 4 + 4) can reveal a heap overflow mitigation.
• The model should own bookkeeping (symbol correlation, string anchoring, structured summaries); humans should own exploitability reasoning and privilege boundary analysis.
• Cost constraints are real: processing entire OS subsystems through large models reaches thousands of dollars quickly. Manual checkpoints aren't just good practice — they're economically necessary.

Slides reference: Slides PDF available (2026-04-04-D2-S1-10-04-macOS-Vulnerability-Research-Augmentin.pdf). Key slides covered the OSS-Sensor tool (github.com/oliviagallucci/oss-sensor), the XNU kernel architecture overview, and the diff-to-hypothesis workflow diagram. Transcript PDF (2026-04-04-Day2-Stage1-Gallucci-txt.pdf) provided detailed technical content.