Reverse Engineering Apple Security Updates
John McIntosh (Security Researcher & Founder · Clearseclabs)
RE//verse 2026 · Day 1 · Coral Ballroom A
Overview
In his compelling RE//verse talk, "Reverse Engineering Apple Security Updates," John McIntosh, a seasoned security researcher and founder of Clearseclabs, unveils a novel approach to dissecting Apple's typically opaque security patches. The presentation introduces agentic diffing, a hybrid methodology that marries traditional binary diffing techniques with the reasoning capabilities of Large Language Models (LLMs) to automate and accelerate root cause analysis of vulnerabilities. This talk is an updated and deepened version of a previous presentation at Objective by the Sea, now incorporating new tooling and a more profound exploration of LLM integration.

Key moments
- 0:30 Introduction to agentic diffing using LLMs in RE
- 3:05 The problem: Apple security updates are opaque
- 4:00 Manual patch diffing workflow for Apple updates
- 6:20 Transitioning from manual analysis to LLM automation
- 11:00 Agentic diffing toolchain and DSPy programming approach
- 16:00 Case study demo: Font Parser CVE root cause analysis
- 20:00 Limitations and challenges of LLM-based RE
- 24:30 Q&A: Workflow execution speed after a patch
Reverse Engineering Apple Security Updates
Speakers: John McIntosh, Security Researcher & Founder, Clearseclabs
Conference: RE//verse
YouTube: https://www.youtube.com/watch?v=4fiKBHuSYMg
Overview
In his compelling RE//verse talk, "Reverse Engineering Apple Security Updates," John McIntosh, a seasoned security researcher and founder of Clearseclabs, unveils a novel approach to dissecting Apple's typically opaque security patches. The presentation introduces agentic diffing, a hybrid methodology that marries traditional binary diffing techniques with the reasoning capabilities of Large Language Models (LLMs) to automate and accelerate root cause analysis of vulnerabilities. This talk is an updated and deepened version of a previous presentation at Objective by the Sea, now incorporating new tooling and a more profound exploration of LLM integration.
McIntosh addresses a critical challenge faced by security researchers and defenders: Apple's security advisories, much like Microsoft's Patch Tuesday, often provide only high-level descriptions of vulnerabilities, leaving the specific technical details and affected code changes ambiguous. Given that Apple's IPSW firmware updates contain thousands of binaries, manually identifying which binary corresponds to a given CVE and then pinpointing the exact code fix is an arduous, intuition-driven task. By demonstrating how LLMs can be leveraged to enhance reverse engineering workflows, McIntosh offers a scalable and repeatable solution to transform these "black box updates into transparent vulnerability insights," marking a significant step forward in automated vulnerability research.
Background
▶ Watch: Introduction to agentic diffing using LLMs in RE (0:30)
The necessity for agentic diffing stems from the inherent opacity of Apple's security updates. When Apple releases a patch, the accompanying CVE advisory often vaguely states the issue, such as "An app may be able to access sensitive user data" due to an "authorization issue." These high-level descriptions leave crucial questions unanswered: Which binaries are affected? Which specific authorization check was modified? What exact code changes were implemented? With IPSW firmware containing thousands of binaries, manually mapping a CVE description to a specific binary or code change is akin to finding a "needle in a haystack."
McIntosh first addressed this problem two years prior at Objective by the Sea, demonstrating a manual process for patch diffing Apple updates. This traditional workflow involved several labor-intensive steps:
- CVE Identification: Pinpointing a CVE of interest.
- Firmware Acquisition: Downloading both the vulnerable and patched versions of the IPSW firmware using the IPSW tool.
- Binary Diffing: Using
IPSW diffto identify all binaries that changed between the two firmware versions, which could number in the thousands. - Manual Mapping: Relying on intuition or simple
grepcommands to correlate the CVE description with the list of changed binaries, thus narrowing down the potential candidates. An example cited was a telephony use-after-free issue from Google Project Zero, where the binary nametelephonydirectly hinted at the relevant file. - File Extraction: Extracting the relevant binaries, often from the
dyld shared cacheif they were dylibs. - Function-Level Diffing: Employing a tool like Ghidra to produce an output detailing added, modified, or deleted functions within the identified binary.
- Root Cause Analysis: Deep-diving into the code changes to understand the vulnerability's fix, often cross-referencing with public reports like those from Project Zero.
While effective, this manual process was time-consuming, highly dependent on the reverse engineer's intuition and experience, and not scalable for the volume of security updates Apple releases. McIntosh's aspiration was to automate this entire workflow, turning it into a "repeatable hands-off workflow" that could fetch CVEs, correlate them with binaries, perform diffing, and generate root cause reports without significant human intervention.
A key shift in McIntosh's approach for this updated talk was the transition from primarily using Ghidra to incorporating Binary Ninja (Binja) and BinDiff for the diffing and analysis phases. He noted significant improvements, including "much faster analysis," an "easier on the eyes" interface, and the ease of developing "working headless scripts on day one" with Binary Ninja, a task that took weeks with Ghidra.
The "gravity of AI" served as the primary catalyst for automating this process. McIntosh's introduction to AI in reverse engineering came in early 2025 with the advent of the Model Context Protocol (MCP), particularly with Anthropic's release. MCP enables LLMs to interact with external tools, providing them with accurate, ground-truth information to reduce hallucinations. A pivotal moment was seeing Lori's (the conference's keynote speaker) Ghidra MCP, which demonstrated LLMs actively and accurately renaming functions, accelerating analysis workflows. McIntosh then developed PyGhidraMCP, a command-line, project-scoped tool, gaining significant experience in managing LLM context windows. He illustrated the power of MCP by showing an LLM making 22 tool calls to trace a Windows application's high-level API call into the kernel, generating a detailed explanation and source code. This experience, combined with his prior manual reverse engineering of Patch Tuesday (which he presented at Recon), solidified his belief in the feasibility of patching at scale through automation, especially with LLMs. The success of the DARPA AIC competition further underscored how LLMs can enable agentic workflows, performing tasks previously achievable only by humans.
The core idea of agentic diffing is to move from a manual, intuition-driven process to an automated one by pairing each manual step with a specialized tool and an LLM agent. Deterministic tools like Binary Ninja, BinDiff, and CVE metadata provide the "ground truth" of code changes and context, while LLMs provide the "reasoning" and "intuition" to connect the dots, making the process scalable and repeatable. McIntosh emphasized that DSPy, a declarative self-improving Python framework, was instrumental in programming these LLMs by defining input/output contracts rather than relying solely on prompting. This approach, combined with React agents (which interleave reasoning with tool use in a loop) and MLflow for introspection and debugging, forms the backbone of his agentic diffing system.
Key Findings
▶ Watch: Manual patch diffing workflow for Apple updates (4:00)
John McIntosh's agentic diffing methodology demonstrates several significant findings regarding the automated reverse engineering of Apple security updates:
- Feasibility of Automated Root Cause Analysis: The talk conclusively proves that it is possible to automate the process of reverse engineering opaque Apple security updates to identify the root cause of vulnerabilities. This transforms a traditionally manual, intuition-driven art into a scalable, repeatable workflow.
- Drastic Reduction of Search Space: LLM agents, particularly the
Correlatoragent, are highly effective at narrowing down the vast number of changed binaries in an IPSW firmware update. McIntosh showcased examples where thousands of updated binaries (e.g., 1,800 or 3,000) were successfully reduced to a mere handful (e.g., 2 or 22 candidates, then to a single primary target), significantly reducing the human effort required for analysis. - Automated Root Cause Report Generation: The system can generate detailed root cause reports that identify specific code changes, such as the introduction of add overflow checks or explicit boundary validations, and pinpoint the relevant functions responsible for the fix. This provides immediate, actionable insights into the nature of the vulnerability.
- Effective Use of LLMs for Contextual Reasoning: LLMs excel at processing natural language CVE descriptions and correlating them with technical details from binary diffs. They can infer relationships and provide plausible explanations even when direct matches are not immediately obvious, as demonstrated by the independent identification of a function name matching a Project Zero call stack.
- Practical Applicability for Research and Defense: The system is not only useful for understanding patched vulnerabilities but also for exploring new research areas, such as using a CVE to dive into complex topics like pointer authentication by identifying relevant binaries and functions for deeper static analysis.
- Cost-Effectiveness: Despite the advanced nature of the analysis, the operational cost of running the LLM-driven process is remarkably low. McIntosh reported that analyzing all CVEs for February 2026 cost approximately $1.50, making it highly accessible for researchers and small teams.
- Achievable Success Rate: While not perfect, the system achieved an approximate 31% success rate in fully identifying the root cause of CVEs processed for February 2026. This percentage, though not 100%, represents a significant automated gain, providing accurate insights for nearly a third of all relevant CVEs with minimal human intervention.
- Scalability Challenges and Limitations: McIntosh candidly addressed the limitations, including issues with hallucinations (LLMs misinterpreting address changes as code fixes), the critical reliance on symbols (lack thereof in kernel extensions causes errors), difficulties with complex bugs requiring kernel debugging, and the inability to detect patches without code changes. Processing large binaries (e.g., 80MB+) also presented challenges.
These findings collectively underscore the transformative potential of integrating LLMs into reverse engineering workflows, offering a powerful, albeit imperfect, tool for navigating the complexities of modern software security.
Technical Deep Dive
▶ Watch: Agentic diffing toolchain and DSPy programming approach (11:00)
The technical core of John McIntosh's agentic diffing system is a sophisticated hybrid system that seamlessly integrates deterministic security tools with intelligent LLM agents. This system is designed to transform the art of manual reverse engineering into a scalable, repeatable, and largely automated workflow.
The toolchain for agentic diffing comprises several key components:
- Data Sources:
- CVE Feeds: The system scrapes Apple's security updates page and the National Vulnerability Database (NVD) to collect detailed CVE information, including ID, impact, and description. This data is crucial for providing context to the LLMs.
- Firmware Downloads: Both the vulnerable and patched versions of Apple's IPSW firmware are downloaded. This is a critical first step, as the firmware packages, often around 13GB, contain all the necessary binaries for diffing.
- Deterministic Tools:
- IPSW: This is the primary tool for interacting with Apple firmware. It's used to download specific IPSW versions (
IPSW download) and to identify all binaries that have changed between two firmware versions (IPSW diff). The output ofIPSW diffis a list of all modified kernel extensions, Mach-Os, and dylibs. - Binary Ninja (Binja) and BinDiff: These are employed for the granular, function-level binary diffing. After
IPSW diffidentifies changed binaries, Binja and BinDiff analyze these files to pinpoint specific added, modified, or deleted functions and their corresponding code changes, often presented as unified diffs. McIntosh noted Binary Ninja's speed and ease of scripting as significant advantages. - Custom Tools: McIntosh developed small, specialized tools to facilitate the workflow, such as a utility to fetch detailed CVE information from NVD and another to read and expose diff information to the LLM at the appropriate stage.
- LLM Agents: The intelligence layer of the system is built upon three distinct LLM agents, orchestrated using DSPy, a declarative self-improving Python framework. DSPy emphasizes "programming LLMs instead of prompting," allowing developers to define input-output contracts and let the LLM fill in the reasoning.
- Platform Classifier: This initial agent takes the CVE description, CWE ID, and references as input. Its task is to predict the target platform (e.g., Apple, Windows, IoT). This is a relatively simple classification task, often straightforward due to direct mentions of "Apple" in the CVE.
- Correlator (CVE-to-Binary Mapper): This agent is critical for addressing the "needle in a haystack" problem. Given a CVE description and the extensive list of changed binaries from
IPSW diff, the Correlator maps the CVE to the most relevant binaries. It uses contextual clues, module names, and library references to score and prioritize binaries, drastically reducing the search space from thousands to a handful. - Diff Analyzer: Once the relevant binaries and their code changes (unified diffs) are identified, the Diff Analyzer agent performs the root cause analysis. It combines the CVE context with the detailed code changes to explain the vulnerability's fix, identifying specific security mitigations (e.g., "add overflow checks," "explicit boundary validations") and relevant functions.
LLM Integration and Workflow:
The system leverages the Model Context Protocol (MCP) to provide LLMs with accurate, ground-truth information from the deterministic tools, thereby reducing hallucinations. McIntosh used React agents, which combine reasoning with tool use in an iterative loop. This means an agent can execute a tool, observe the output, reason about it, and decide on the next tool to run, or the next step to take, to achieve its goal. MLflow is integrated for introspection, allowing developers to debug the agent's decision-making process by inspecting every step within the loop.
The end-to-end workflow, integrating deterministic and agentic steps, is as follows:
- CVE Fetching: Automatically retrieve CVE details from Apple advisories and NVD.
- Platform Classification: The
Platform Classifierdetermines the target operating system. - Device Selection: Based on the CVE, the appropriate device (e.g., iPhone, Mac) is selected to guide firmware download.
- Firmware Download & Diff:
IPSWdownloads the vulnerable and patched firmwares and identifies all changed binaries. - Binary Correlation: The
Correlatoragent takes the CVE description and the list of changed binaries to identify the most probable targets. - Function-Level Diffing: Binary Ninja and BinDiff perform detailed analysis on the correlated binaries, producing diffs of function-level changes.
- Root Cause Analysis: The
Diff Analyzeragent, armed with the CVE context and specific code diffs, generates a comprehensive report explaining the vulnerability's fix.
Challenges and Limitations:
McIntosh highlighted several practical challenges. The absence of symbols in kernel extensions can mislead LLMs, as they expect symbols for proper analysis. Hallucinations remain a concern; LLMs might misinterpret non-functional changes, like address offsets due to recompilation, as critical code fixes. Some complex bugs might exceed the current capabilities of LLMs, requiring kernel debugging for full verification. Furthermore, patches not involving code changes (e.g., configuration updates) cannot be detected by a diffing approach. Finally, the sheer size of some binaries (e.g., 80MB) can pose processing challenges for diffing tools and LLMs. Despite these, the system's ability to handle large context windows (e.g., Gemini's 1 million tokens) at a low cost (around $1.50 for a month's worth of CVEs) demonstrates its significant potential.
Demo / Proof of Concept
▶ Watch: Case study demo: Font Parser CVE root cause analysis (16:00)
John McIntosh provided a live demonstration and detailed case studies of his agentic diffing system, showcasing its capabilities through a lightweight framework built with Chainlit.
Case Study 1: Alabama CVE (Font Parser)
This case study began with a real-world scenario: McIntosh saw a tweet from Patrick Wardle about an Alabama CVE and decided to run it through his system. The process, from initial input to a full report, took approximately 2.5 hours, with the actual report generation taking only a few minutes after the initial data curation.
- CVE Details Fetching: The system automatically pulled detailed information about the CVE.
- Platform Classification: The
Platform Classifiereasily identified the platform as Apple due to direct references in the CVE description. - Device Selection: The system chose iPhone as the target device.
- Binary Identification: Since it was an out-of-band patch,
IPSW difffound only 22 binaries that had changed. - Binary Correlation: The
Correlatoragent analyzed these 22 candidates. It identified Font Parser as the most relevant binary, scoring it highly while other binaries received zero scores. The reasoning provided by the LLM included keywords like "font parser," "module," and "lexical library." - Function-Level Diffing: Binary Ninja and BinDiff then analyzed Font Parser, identifying 10 changed functions.
- Root Cause Report: The
Diff Analyzeragent generated a concise root cause report, stating that the "patch primarily introduces add overflow checks, explicit boundary validations." The report included snippets of the actual code changes and highlighted the relevant functions, providing a complete, end-to-end breakdown of the fix.
Case Study 2: Dylan's Double Free (1,800 binaries)
This example showcased the system's ability to handle a much larger search space and provide valuable insights even when a full root cause isn't immediately found.
- Platform & Device: The system classified the platform as Apple and selected Mac as the device.
- Binary Identification:
IPSW diffrevealed a staggering 1,800 binaries updated in this firmware. - Binary Correlation: The
Correlatoragent, after running its React loop, successfully narrowed this down to just two highly scored binaries as plausible candidates. - Root Cause Hints: In this instance, the
Diff Analyzerdid not pinpoint the exact root cause but provided strong "hints" and "plausible places" for the vulnerability. Crucially, one of the function names identified by the LLM independently matched a call stack mentioned in the Project Zero report for the bug, demonstrating the agent's contextual reasoning capabilities. The bug's original author, Dylan, later verified this analysis. This case highlighted the system's power in reducing a search space of 3,000 updated binaries to just two, even if a full root cause wasn't autonomously identified. McIntosh also noted that sending "100 plus unified diffs" to a local LLM was "way too much," requiring a more powerful, likely cloud-based, model for effective processing.
Case Study 3: Pointer Authentication (Research)
McIntosh briefly touched upon how the system could be used for exploratory research. Instead of focusing on a specific CVE fix, researchers could use the same process to identify relevant binaries and functions related to a new area, such as pointer authentication. Once identified, the researcher could then "dive deeper" into Binary Ninja for static analysis, pivoting from the automated findings to manual exploration.
Performance and Logistics:
McIntosh acknowledged that while the analysis itself is fast (minutes for a report), the initial data curation, specifically downloading and extracting large IPSW firmwares (around 13GB), can be a logistical challenge. He estimated that with the right machine, this could be done within an hour, though he personally runs these processes one-by-one and checks the results the next day. The overall efficacy of these demonstrations underscores the system's potential to significantly streamline vulnerability research.
Defensive Implications
▶ Watch: Q&A: Workflow execution speed after a patch (24:30)
The agentic diffing methodology presented by John McIntosh offers profound implications for security defenders, transforming how organizations can approach Apple security updates and proactive threat intelligence.
- Rapid Vulnerability Triage and Assessment: Defenders can move beyond Apple's opaque, high-level CVE advisories. By automating the root cause analysis, security teams can swiftly gain a detailed understanding of the actual code changes, the specific binaries affected, and the nature of the fix. This enables much faster and more accurate triage of newly disclosed vulnerabilities, allowing organizations to assess their exposure and prioritize remediation efforts more effectively.
- Targeted Patch Verification and Mitigation: With precise knowledge of the code changes, defenders can verify that patches have been correctly applied and are indeed effective. This is crucial in environments where patch deployment might be complex or where custom mitigations are required. Furthermore, if a patch is delayed or unavailable, understanding the exact vulnerability allows for the development of highly targeted interim mitigations or detection rules.
- Enhanced Threat Intelligence and Exploit Detection: Detailed root cause analysis provides invaluable threat intelligence. By understanding how Apple fixed a vulnerability, defenders can better anticipate future attack vectors, identify similar weaknesses in other parts of the codebase, or develop more robust detection signatures for potential exploits. This knowledge empowers security operations centers (SOCs) to improve their endpoint detection and response (EDR) capabilities.
- Scalable Security Research: The automation drastically reduces the manual effort associated with reverse engineering security updates. This means that even smaller security teams or individual researchers can perform in-depth analysis across a broader range of CVEs, keeping pace with Apple's frequent and numerous patch cycles without requiring an army of highly specialized reverse engineers. The low operational cost (approximately $1.50 for a month's worth of CVEs) makes advanced analysis more accessible.
- Proactive Vulnerability Discovery: As McIntosh noted in his roadmap, the insights gained from understanding how vulnerabilities are patched can be fed into coding agents to find similar bugs. For defenders, this could evolve into a capability to proactively scan their own codebases (if applicable) or monitor third-party components for similar vulnerability patterns before they are exploited.
- Addressing "Black Box" Challenges: Apple's closed ecosystem traditionally makes it difficult for external researchers to deeply understand security updates. Agentic diffing provides a powerful lens into this "black box," fostering greater transparency and allowing defenders to make more informed decisions rather than relying solely on high-level vendor advisories.
However, defenders must also be aware of the system's limitations:
- Verification of LLM Outputs: While powerful, LLMs can hallucinate or misinterpret changes (e.g., address shifts due to recompilation). Defenders must implement a robust verification step for critical findings, potentially involving manual review by an experienced reverse engineer or dynamic analysis.
- Symbol Absence Impact: Analysis of kernel extensions or other stripped binaries will be less precise due as the lack of symbols limits the LLM's context. Defenders should account for higher uncertainty in these areas.
- Undetected Non-Code Fixes: The system relies on code diffs. Patches implemented through configuration changes, policy updates, or other non-code modifications will not be detected or explained by this methodology.
- Complexity Threshold: Extremely complex or novel vulnerabilities may still require significant manual reverse engineering and kernel debugging beyond what the automated system can currently provide.
In essence, agentic diffing serves as a force multiplier for defenders, equipping them with unprecedented speed and depth in understanding Apple's security landscape, allowing them to shift from reactive patching to a more proactive and informed defensive posture.
Key Takeaways
Here are the main points from John McIntosh's talk on agentic diffing for Apple security updates:
- Agentic Diffing is a Transformative Approach: This methodology effectively combines deterministic reverse engineering tools (like IPSW, Binary Ninja, and BinDiff) with intelligent LLM agents (orchestrated via DSPy and React patterns) to automate the analysis of Apple's opaque security updates, turning a manual "art" into a scalable, repeatable workflow.
- Efficient CVE-to-Binary Mapping: LLMs excel at correlating high-level CVE descriptions with lists of thousands of changed binaries, drastically reducing the search space to a handful of relevant files. This crucial step replaces manual intuition and significantly accelerates the start of deep technical analysis.
- Automated Root Cause Reporting: The system can generate detailed root cause reports, identifying specific code changes (e.g., "add overflow checks," "explicit boundary validations") and the functions responsible for the fix. This provides immediate, actionable insights into the nature of the vulnerability and its resolution.
- Practical and Cost-Effective: The framework is demonstrated to be practical for real-world scenarios, processing CVEs in minutes (after firmware preparation). Despite leveraging advanced AI, the operational cost is remarkably low, with a month's worth of CVE analysis costing approximately $1.50.
- Challenges and Caveats Exist: While powerful, the system faces limitations including LLM hallucinations (misinterpreting address changes), critical reliance on symbols (kernel extensions often lack them), and difficulties with highly complex bugs or patches that don't involve code changes. Human verification remains a necessary step for critical findings.
- Future Potential Beyond Patch Analysis: The insights gained from agentic diffing can be extended beyond merely understanding patches. McIntosh highlighted its potential for exploring new research areas (e.g., pointer authentication) and for feeding context into coding agents to proactively discover new vulnerabilities, hinting at a broader impact on security research.
About the Speaker(s)
John McIntosh is an experienced Security Researcher with over a decade in the field, renowned for his work in reverse engineering. He is the Founder of Clearseclabs, a company focused on security research, and is also an active open source developer, contributing many of his tools to GitHub. McIntosh is a frequent blogger, using writing as a means to deepen his understanding and share knowledge, and occasionally a speaker at prominent security conferences. He has previously presented his work at Objective by the Sea and Recon. McIntosh's particular expertise and passion lie in Apple security updates, a domain he humorously suggests he was "designed to work on." His presentation at RE//verse showcases his innovative integration of LLMs into complex reverse engineering workflows, building on his prior work with tools like PyGhidraMCP.
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
McIntosh actually built a pipeline that does something useful with LLMs instead of just slapping a chat wrapper on a script. Using MCP and DSPy to chain Binary Ninja, BinDiff, and ipsw into a headless patch-diffing engine for Apple updates is exactly what this industry needs. It's a massive time-saver for vulnerability researchers tired of playing needle-in-a-haystack with Apple's notoriously vague advisories.
Heather Calloway (CISO) — STRONG ACCEPT
McIntosh demonstrates an LLM-driven method for reverse-engineering Apple's opaque security patches at scale for pennies. While the mechanics belong to the reverse engineering team, the output is pure threat intelligence that changes how quickly organizations can build targeted detections.