LLMxCPG: Context-Aware Vulnerability Detection Through Code Property Graph-Guided Large Language Models

My ZIP isn't your ZIP: Identifying and Exploiting Semantic Gaps Between ZIP Parsers

Speakers: Yufan You, Jianjun Chen, Qi Wang, Haixin Duan (Tsinghua University; Zhongguancun Laboratory)

Conference: USENIX Security

YouTube: https://www.usenix.org/system/files/usenixsecurity25-you.pdf (Note: This is a peer-reviewed paper, not a recorded talk. The link provided is to the paper PDF.)

Overview

The ubiquitous ZIP file format, a foundational component for everything from office documents and Android applications to Java archives and browser extensions, harbors a pervasive and under-explored security vulnerability: semantic gaps between its numerous parsing implementations. This paper, "My ZIP isn't your ZIP: Identifying and Exploiting Semantic Gaps Between ZIP Parsers," by Yufan You, Jianjun Chen, Qi Wang, and Haixin Duan from Tsinghua University and Zhongguancun Laboratory, presents a groundbreaking systematic study into these inconsistencies. The research unveils how seemingly minor differences in how various software interprets the ZIP specification can be weaponized by attackers to bypass critical security measures.

The authors introduce ZIPDIFF, a novel differential fuzzer designed to systematically uncover these parsing discrepancies. Through an extensive evaluation involving 50 ZIP parsers across 19 programming languages, ZIPDIFF revealed that nearly every pair of parsers is susceptible to some form of ambiguity. The study categorizes these findings into 14 distinct types of parsing ambiguities, with 10 of these being newly identified or significantly extended. The practical impact is demonstrated through five real-world exploitation scenarios, including bypassing secure email gateways, spoofing office document content, impersonating VS Code extensions, and tampering with signed nested JAR files. These findings underscore a critical need for more robust and consistent handling of the ZIP format, leading to responsible disclosures, bug bounties from major vendors like Gmail, Coremail, and Zoho, and the assignment of three CVEs (CVE-2024-24789, CVE-2024-7788, CVE-2024-38807).

This research is significant because it moves beyond ad-hoc discovery of individual ZIP vulnerabilities to provide a systematic and comprehensive analysis of parsing ambiguities. By meticulously dissecting the ZIP format's imprecisely specified elements and the diverse interpretations across implementations, the paper offers a robust framework for understanding and mitigating a class of vulnerabilities that have broad implications for software supply chain security, data integrity, and communication security. The proposed mitigation strategies and insights into better file format design provide valuable guidance for developers and standards bodies alike.

Background

The ZIP file format, originally conceived by PKWARE in 1989 and specified in APPNOTE.txt, has evolved into one of the most widely adopted archive formats. Beyond its common .zip extension, it serves as a fundamental container for a myriad of other file types, including OOXML (e.g., DOCX, XLSX), ODF (e.g., ODT), Java Archives (JAR), Android Packages (APK), Visual Studio Extensions (VSIX), and browser extensions (CRX, XPI). Its pervasive use means that any vulnerabilities within its parsing mechanisms can have far-reaching security implications across diverse applications and platforms.

A typical ZIP file is structured into three main components: Local File Entries (LFHs), the Central Directory (CD), and the End Of Central Directory Record (EOCDR). Each LFH contains metadata and the compressed file data. The CD comprises Central Directory Headers (CDHs), each pointing to a corresponding LFH. The EOCDR, located at the end of the file, provides pointers to the central directory. A key characteristic of the ZIP format is the redundancy of metadata fields, such as filename, compression method, and file sizes, which are often stored in both the LFH and CDH. While this redundancy aids in data recovery and enables streaming data processing, it also creates fertile ground for ambiguities when these redundant fields contain conflicting information.

ZIP files can be parsed in two primary modes: the standard mode and the streaming mode. The standard mode typically begins by locating the EOCDR from the end of the file, then navigates the central directory via CDHs to find the LFHs and their associated data. In contrast, the streaming mode reads LFHs sequentially from the beginning of the file, either ignoring the central directory or performing consistency checks later. The format has also undergone extensions like ZIP64, introduced to support files larger than 4GB and more than 65535 entries, further adding to its complexity.

The root cause of these semantic gaps lies in the ZIP format's original specification. The APPNOTE.txt was "casually written," leaving many critical details open to interpretation. While ISO/IEC 21320-1 attempted to formalize the standard in 2015, it primarily imposed restrictions rather than clarifying existing ambiguities. This lack of precise definition, coupled with the sheer volume of independent implementations across various programming languages and applications, has prevented the emergence of a single "de facto standard." Furthermore, many developers adhere to Postel's Law ("be conservative in what you do, be liberal in what you accept from others"), leading parsers to attempt resolving malformed files rather than simply rejecting them, which exacerbates behavioral discrepancies.

Prior research has identified individual instances of ZIP parsing ambiguity vulnerabilities, such as CVE-2003-1154 for antivirus bypass and the infamous Android master key vulnerability [20] which exploited a mismatch between signature verification and decompression components. Other studies have focused on related ZIP vulnerabilities like Path Traversal (ZIP Slip) [38] and Denial of Service (ZIP bomb) attacks. However, these efforts largely relied on manual, ad-hoc discovery methods. This paper addresses a significant gap by conducting the first systematic study of ZIP parsing ambiguities, providing a comprehensive classification and evaluation of their prevalence and exploitable nature.

Key Findings

The research conducted using ZIPDIFF yielded several critical findings that highlight the pervasive and exploitable nature of ZIP parsing ambiguities:

• Prevalence of Inconsistencies: The evaluation against 50 ZIP parsers across 19 programming languages demonstrated that parsing inconsistencies are remarkably common. A staggering 1221 out of 1225 possible pairs of parsers were found to be vulnerable to at least one type of ambiguity. This indicates that almost any interaction between two different ZIP implementations can lead to divergent interpretations of a crafted archive.
• Systematic Classification of Ambiguities: The authors systematically classified the identified inconsistencies into 14 distinct types, grouped into three major categories: Redundant Metadata, File Path Processing, and ZIP Structure Positioning. This classification provides a much-needed framework for understanding the diverse root causes of these vulnerabilities.
• Discovery of Novel Ambiguities: Out of the 14 identified types, 10 were either newly discovered or significantly extended with novel variants and techniques to cause inconsistencies and bypass existing checks. This demonstrates the effectiveness of the ZIPDIFF differential fuzzing approach in uncovering previously unknown attack vectors.
• Real-world Exploitation Scenarios: The paper provides concrete evidence of the practical impact of these ambiguities through five real-world exploitation scenarios. These include bypassing secure email gateways, spoofing content in office documents, forging signatures in LibreOffice documents and Spring Boot nested JAR files, and impersonating VS Code extensions. These scenarios targeted widely used products and services, showcasing the broad attack surface.
• Significant Vendor Impact and Disclosure: The research led to successful responsible disclosures to numerous affected vendors. This includes bug bounty rewards from Gmail ($1337), Coremail (approx. $400), and Zoho ($200), as well as acknowledgments from Outlook, Proton Mail, Naver, mail.com, and mail.ru. Crucially, three CVEs were assigned: CVE-2024-24789 for Go's archive/zip package, CVE-2024-7788 for LibreOffice's signature forgery vulnerability, and CVE-2024-38807 for Spring Boot's nested JAR signature forgery.
• Proposed Mitigation Strategies: The paper concludes by proposing seven practical mitigation strategies, offering actionable advice for developers and organizations to defend against these identified issues.

These findings collectively underscore the critical security implications of semantic gaps in widely used file formats and emphasize the necessity for systematic research and improved specification clarity.

Technical Deep Dive

The core of this research lies in ZIPDIFF, a mutation-based blackbox differential fuzzer specifically designed to identify inconsistencies between ZIP parsers. Its workflow, illustrated in Fig. 3 of the paper, involves generating initial well-formed ZIP files, mutating them, feeding them to multiple parsers, analyzing their outputs, and using this feedback to guide further mutations.

ZIPDIFF Design and Implementation

1. Corpus Generation: ZIPDIFF begins by generating an initial corpus of valid ZIP files. These samples are constructed by randomly selecting filenames, contents, compression methods, and other ZIP parameters.
2. Mutation Strategies: To maximize the chances of triggering ambiguities, ZIPDIFF employs two main types of mutation strategies:

• ZIP-Level Mutations: These are grammar-aware and semantic-aware mutations that target specific fields within the ZIP file structure. The fuzzer incorporates 46 such strategies, designed to mutate individual fields or combinations of fields while attempting to maintain structural integrity (e.g., repairing inter-field dependencies like sizes and offsets). Examples include modifying compression methods, altering size fields, or manipulating filename components.
• Byte-Level Mutations: To explore less structured edge cases, ZIPDIFF also applies generic byte-level mutations such as insertion, deletion, modification, duplication, splicing, and bit flipping. These mutations can corrupt the ZIP structure, but are crucial for uncovering unexpected parser behaviors.

1. Difference Analyzer: After mutation, each test sample is fed to the collected ZIP parsers. The parsers are instructed to extract the archive onto the file system.

• Output Hashing: For successful extractions, a hash value of the output directory is computed. To focus on exploitable inconsistencies, the hashing process specifically ignores invalid characters in filenames (which often show non-exploitable discrepancies) and empty directories.
• Inconsistency Definition: Two parsers are considered "inconsistent" if they both successfully extract a sample file, but their computed output directory hash values differ. Cases where one parser succeeds and another fails are not classified as inconsistencies, as practical exploitation often requires both components to "successfully" process the file, albeit differently.
• Interesting Sample Detection: Samples are deemed "interesting" if they introduce new inconsistent parser pairs or successfully parsed files compared to existing seeds in the corpus, guiding the fuzzer towards novel ambiguity-triggering inputs.

1. Mutation Strategy Selection: To balance exploration of new mutation types and exploitation of effective ones, ZIPDIFF treats mutation strategy selection as a multi-armed bandit problem. It utilizes a modified Upper Confidence Bounds (UCB) formula (Equation 1 in the paper) to weigh strategies based on their historical success in generating interesting samples. Instead of a deterministic argmax selection, ZIPDIFF uses a softmax function (Equation 2) with a temperature parameter (β) to introduce randomness and prevent premature convergence on a single strategy. Furthermore, a decay rate (α) is applied to usage counts and rewards, giving more weight to recent evaluation results and adapting to changes in corpus effectiveness over time.

Classification of ZIP Parsing Ambiguities

The 14 distinct types of ambiguities discovered by ZIPDIFF are categorized as follows:

1. Redundant Metadata: These ambiguities arise from conflicting information stored in multiple locations within the ZIP file.

• A1: Compression Method Confusion: Parsers disagree on the compression method (e.g., compressed data explicitly marked as "stored" in LFH/CDH).
• A2: File Size Confusion: Inconsistencies arise from multiple sources of file size information (LFH, CDH, data descriptors, ZIP64 extended fields) or when fields are set to special values (0xFFFFFFFF for ZIP64, 0 for data descriptors) that parsers might misinterpret. The non-cryptographic nature of CRC32 checksums further aids exploitation by allowing data manipulation without detection.
• A3: Filename Confusion: Discrepancies in how parsers interpret filenames, particularly concerning the Info-ZIP Unicode path extra field (UP). Ambiguities include selection among multiple UPs, handling of UP version fields, name CRC32 checks, and interaction with the new language encoding flag. Six specific edge cases were identified.
• A4: Fake Directory: Parsers disagree on whether a file entry represents a regular file or a directory, based on path termination (slashes/backslashes) or conflicting values in the external file attributes field in the CDH (which is host-system dependent).
• A5: Fake Encryption: Parsers interpret encryption flags inconsistently, leading one parser to treat unencrypted data as encrypted (and thus fail to extract) while another processes it successfully. Some parsers may even abandon processing the entire archive upon encountering an "encrypted" file.

1. File Path Processing: These ambiguities relate to how parsers interpret and resolve file paths within the archive.

• B1: Duplicate Files: When multiple files share the same path, different parsers may have divergent policies on which file to select (e.g., first, last, or error).
• B2: Invalid Characters: Parsers handle invalid characters in file paths (e.g., ASCII control characters, invalid Unicode, Windows-specific characters like *, <, >) inconsistently. Some remove, others replace, and null bytes can prematurely terminate strings.
• B3: Path Canonicalization: Different parsers may canonicalize file paths inconsistently, leading them to believe content.xml and ./content.xml are either the same or different files. This includes handling redundant slashes (//), backslashes, and dot segments (., ..).
• B4: Case Sensitivity: Some parsers (especially on Windows) treat file paths case-insensitively, while others are case-sensitive, leading to different interpretations of duplicate files like FILE.TXT and file.txt.

1. ZIP Structure Positioning: These ambiguities occur when parsers disagree on the fundamental layout and location of ZIP structures within the file.

• C1: Streaming Parsing: Discrepancies between standard (EOCDR-driven) and streaming (sequential LFH processing) modes. This category includes scenarios like LFHs without corresponding CDHs, truncated LFH streams, LFH desynchronization (LFH appearing where streaming expects data), and ambiguities in locating the data descriptor.
• C2: EOCDR Selection: When multiple EOCDR signatures are present (e.g., within comment fields), parsers may select different EOCDRs. Factors include backward scanning, comment length consistency checks, and unique heuristics (e.g., libzip's "consistency" score).
• C3: CDH Count Confusion: Conflicts in determining the number of Central Directory Headers. The EOCDR provides total and current disk CDH counts, size, and position, all of which can conflict. Some parsers read all CDHs regardless of count fields, others respect the central directory size, and the 16-bit limits for CDH counts can lead to modulo arithmetic interpretations (e.g., 1 vs. 65537 entries).
• C4: CD & LFH Offset Confusion: Discrepancies in locating the central directory and LFHs. Parsers may assume the EOCDR immediately follows the central directory, or incorrectly adjust offsets for prepended data common in self-extracting archives.
• C5: ZIP64 EOCD Processing: Ambiguities specific to the ZIP64 extension. This includes how the ZIP64 End Of Central Directory Locator (ZIP64 EOCDL) and ZIP64 End Of Central Directory Record (ZIP64 EOCDR) are located (fixed offset vs. signature search), when they should be used over regular EOCDRs, and how fields from both might be mixed. ZIP64 EOCDRs can also inherit C3 and C4 issues.

Ablation Study

An ablation study confirmed the effectiveness of ZIPDIFF's design choices. The "Full Setup" (using softmax-based UCB and both ZIP-level and byte-level mutations) significantly outperformed setups using "Argmax-Based UCB" or "Byte Mutation Only" in terms of the median number of inconsistent parser pairs discovered over time (1197 vs. 1183 and 1055 respectively over 24 hours). This validates the importance of sophisticated mutation strategies and balanced exploration/exploitation in fuzzing.

Demo / Proof of Concept

As this research is presented as a peer-reviewed paper rather than a live talk, there isn't a traditional "demo" or "proof of concept" video. Instead, the paper meticulously details five real-world exploitation scenarios where the identified parsing ambiguities were weaponized against widely used software and services. These scenarios serve as concrete proofs of concept for the broad impact of ZIP parsing inconsistencies.

1. Secure Email Gateway Bypass:

• Target: Antivirus scanners in email products (e.g., Gmail, Coremail, Zoho, Outlook, Proton Mail, Naver, mail.com, mail.ru, iCloud, inbox.lv) versus common ZIP unarchivers (e.g., WinRAR, 7-Zip, Info-ZIP).
• Mechanism: Attackers craft a malicious ZIP file containing malware. Due to ambiguities like File Size Confusion (A2), Fake Directory (A4), or Fake Encryption (A5), the email gateway's antivirus scanner might interpret the malware as truncated, uncompressed (when it's actually compressed), a directory, or an encrypted file it cannot process. Consequently, the antivirus deems the attachment safe. However, the victim's local ZIP unarchiver, parsing the file differently (e.g., using correct LFH sizes, recognizing it as a file, or handling encryption differently), successfully extracts and executes the malware.
• Impact: All tested email products were vulnerable to some construction method, demonstrating a critical failure in perimeter defenses. Gmail, Coremail, and Zoho awarded bounties for these findings.

1. Office Document Content Spoofing:

• Target: Inconsistencies between different office applications (e.g., Microsoft Office, LibreOffice, WPS Office) and online document processors (e.g., plagiarism checkers like PapersOwl, Grammarly, CNKI, or AI assistant services).
• Mechanism: Office documents (OOXML, ODF) are ZIP files containing XML content (e.g., word/document.xml). Attackers create documents where, due to ambiguities like Case Sensitivity (B4), Duplicate Files (B1), or ZIP64 EOCD Processing (C5), the content displayed by a local office suite differs from what an online service perceives.
• Case Study 1: CNKI plagiarism checker located XML files case-insensitively and picked the last duplicate, while LibreOffice ignored uppercase WORD/DOCUMENT.XML. WPS Office located files case-insensitively but picked the first duplicate.
• Case Study 2: Exploiting ZIP64 EOCD Processing (C5), a document was crafted where Microsoft Office, LibreOffice, and WPS Office ignored a ZIP64 EOCDR and displayed plagiarized content, while PapersOwl and Grammarly (using streaming parsing, C1) recognized the ZIP64 EOCDR and saw the non-plagiarized version.

1. LibreOffice Document Signature Forgery:

• Target: Parsing inconsistencies within LibreOffice, specifically between its signature verifier and document viewer.
• Mechanism: LibreOffice's parser operates in a "normal mode" for signature verification and a "recovery mode" for display when corruption is detected. An attacker can modify a legitimately signed document by exploiting Filename Confusion (A3) (e.g., altering the LFH filename while keeping the CDH filename for signature validation). The signature verifier in normal mode validates against the original, legitimate content (based on CDH filename), while the document viewer in recovery mode displays the manipulated content (based on the LFH filename).
• Impact: This allowed forging signed documents, leading to CVE-2024-7788 for LibreOffice.

1. Spring Boot Nested JAR Signature Forgery:

• Target: Inconsistencies between two JAR parsers used by Spring Boot Loader's NestedJarFile class.
• Mechanism: Spring Boot's custom ZipContent parser operates in the standard ZIP parsing mode, while Java's JarInputStream (used for signature verification) operates in the Streaming Parsing (C1) mode. By exploiting this difference, an attacker could craft a nested JAR file where arbitrary malicious content is inserted, yet it still passes signature verification by JarInputStream.
• Impact: This vulnerability allowed tampering with signed nested JAR files, resulting in CVE-2024-38807 for Spring Boot.

1. VS Code Extension Impersonation:

• Target: Parsing inconsistencies between the VS Code extension Marketplace server and the VS Code client.
• Mechanism: VS Code extension packages are ZIP files, and the extension.vsixmanifest file contains critical metadata including the publisher and extension ID. Exploiting Filename Confusion (A3), specifically through the Unicode path extra field, an attacker could craft a package that the Marketplace server identifies with their legitimate namespace (e.g., attacker.bar), but the VS Code client interprets as impersonating a target extension (e.g., bob.foo).
• Impact: This circumvents namespace isolation rules, allowing attackers to replace or impersonate existing, trusted extensions. Open VSX acknowledged the report and implemented checks.

These real-world examples unequivocally demonstrate that ZIP parsing ambiguities are not theoretical curiosities but rather critical vulnerabilities with tangible security consequences across a wide array of software ecosystems.

Defensive Implications

The systematic identification of ZIP parsing ambiguities necessitates a multi-faceted approach to defense. The authors propose seven mitigation strategies, each with its own trade-offs and applicability:

1. Use the Same Parser: The most straightforward solution is to ensure that all components in a workflow processing a ZIP file utilize the exact same parser implementation. This eliminates semantic gaps by enforcing a single interpretation. However, this is often impractical in real-world scenarios where different parties control different parts of a workflow, or when using third-party libraries.
2. On-Access Scanning: This strategy, commonly employed by antivirus software, involves deferring full content scanning until the archive is extracted and its contents are accessed. This compensates for the limitations of scanning the archive itself by verifying the actual extracted files. The principle can be extended to other scenarios, where a component uses the parsing result of another component rather than re-parsing the ZIP file.
3. Normalize the ZIP File: Since exploitation often relies on carefully crafted, malformed ZIP files, a simple yet effective mitigation is to normalize the archive. This involves extracting the ZIP file's contents and then repacking them into a new, well-formed ZIP archive before further processing. This process typically resolves most ambiguities by creating a standard, unambiguous structure. This strategy is suitable when only the contents are important, not the integrity of the original ZIP file structure.
4. Identify Ambiguous Patterns in ZIP Files: Parsers can be enhanced to actively detect patterns indicative of malformed or ambiguous ZIP files, such as unused bytes, conflicting metadata fields (CDH vs. LFH), or multiple EOCDRs. Libraries like libzip offer CHECKCONS flags for intensive consistency checks. While effective against known ambiguities, this approach is limited by the attacker's ability to craft novel, undetected patterns. Furthermore, legitimate use cases (e.g., self-extracting archives with prepended executables, APKs with signature data before the central directory) might be flagged as malformed, leading to false positives.
5. Incorporate Different Parsing Logics: To detect previously unknown ambiguities, a service could employ multiple parsers and compare their outputs for consistency. If discrepancies are found, the file can be rejected or flagged. This is resource-intensive, requiring significant computing power to extract archives multiple times. An alternative is to design a single robust parser that incorporates multiple parsing logics or exhaustively tries all possible interpretations to identify and report ambiguities internally. Gmail, for instance, identifies all CDHs and their corresponding files, even if they overlap or are outside the central directory.
6. Fix Unique Parsing Behaviors: Given the vague ZIP specification, establishing a single "correct" behavior can be challenging. However, addressing outlier behaviors, where a few parsers deviate significantly from the majority, can improve consistency. The authors' work facilitated such fixes, for example, libzip aligning its EOCDR selection policy. This strategy aims to reduce the attack surface by making parser behaviors more predictable.
7. Better File Format Design: For future or redesigned archive formats, the lessons learned from ZIP's ambiguities are invaluable:

• Unambiguous Location: Every part of the format should be unambiguously located, avoiding reliance on fragile signature searching.
• Clear Conflict Resolution: Redundant data fields should be minimized or, if necessary, have clearly defined rules for conflict resolution.
• Backward Compatibility: Design for extensions should explicitly define whether a feature is enabled or not to avoid implicit interpretations.
• Security-Sensitive Data: Fields that can be silently ignored should not contain security-sensitive information (e.g., filenames or sizes in extra fields).

By implementing a combination of these strategies, developers and organizations can significantly enhance the security posture of systems that rely on ZIP file processing, thereby reducing the window for exploitation via semantic gaps.

Key Takeaways

• Pervasive Vulnerability: ZIP parsing ambiguities are not isolated incidents but a widespread problem, with almost all pairs of real-world ZIP parsers exhibiting inconsistencies in their interpretation of crafted ZIP files.
• Systematic Discovery: The ZIPDIFF differential fuzzer effectively and systematically identified 14 distinct types of ZIP parsing ambiguities, including 10 previously unknown or significantly extended variants.
• Broad Real-world Impact: These ambiguities can be weaponized in critical real-world attack scenarios, such as bypassing secure email gateways, spoofing office document content, forging digital signatures, and impersonating software extensions.
• Fundamental Security Problem: The research highlights semantic gaps as a fundamental security challenge stemming from imprecise specifications and diverse implementations, extending beyond just the ZIP format.
• Actionable Mitigations: A range of defensive strategies, from using consistent parsers and normalizing files to designing more robust file formats, are available to address these vulnerabilities.
• Successful Disclosure and Recognition: Responsible disclosure efforts led to significant vendor acknowledgments, multiple bug bounties (e.g., Gmail, Coremail, Zoho), and the assignment of three CVEs (CVE-2024-24789, CVE-2024-7788, CVE-2024-38807), underscoring the practical importance of this research.

About the Speaker(s)

The research paper "My ZIP isn't your ZIP: Identifying and Exploiting Semantic Gaps Between ZIP Parsers" was authored by Yufan You, Jianjun Chen, Qi Wang, and Haixin Duan. All authors are affiliated with Tsinghua University and Zhongguancun Laboratory. Jianjun Chen is noted as the corresponding author. Their work focuses on software security, particularly in identifying and exploiting vulnerabilities arising from semantic gaps in widely used file formats and network protocols. Their contributions to this paper demonstrate a deep expertise in differential fuzzing techniques and a rigorous approach to uncovering systemic security weaknesses in fundamental software components.