Two Bytes to Rule Adobe Reader Twice: The Black Magic Behind the Byte Order Mark
Ke Liu (Senior Security Researcher · Tencent Security Xuanwu Lab)
ZeroNights 2019 · Day 1
Overview
In his compelling ZeroNights presentation, "Two Bytes to Rule Adobe Reader Twice: The Black Magic Behind the Byte Order Mark," Ke Liu, a Senior Security Researcher at Tencent Security Xuanwu Lab, unveiled a critical class of vulnerabilities impacting Adobe Acrobat and Reader. The talk meticulously detailed how a fundamental misunderstanding and incorrect handling of string types, specifically involving the Byte Order Mark (BOM), led to widespread out-of-bounds memory access issues. These vulnerabilities, which existed across all versions of Adobe Acrobat Reader DC since April 2015, enabled attackers to achieve significant security compromises, including information disclosure and, in some cases, remote code execution.

Key moments
- 0:00 Talk introduction and speaker background
- 8:30 Understanding the Byte Order Mark (BOM)
- 16:00 Adobe's string handling flaws and type confusion
- 20:30 CVE-2019-7032: Out-of-bounds read for info disclosure
- 27:00 Code execution via out-of-bounds read/write vulnerability
- 31:30 Advanced exploit techniques for code execution
- 36:30 Lessons learned and secure coding suggestions
Two Bytes to Rule Adobe Reader Twice: The Black Magic Behind the Byte Order Mark
Speakers: Ke Liu, Senior Security Researcher, Tencent Security Xuanwu Lab
Conference: ZeroNights
YouTube: https://www.youtube.com/watch?v=rBI3cmSbsu0
Overview
In his compelling ZeroNights presentation, "Two Bytes to Rule Adobe Reader Twice: The Black Magic Behind the Byte Order Mark," Ke Liu, a Senior Security Researcher at Tencent Security Xuanwu Lab, unveiled a critical class of vulnerabilities impacting Adobe Acrobat and Reader. The talk meticulously detailed how a fundamental misunderstanding and incorrect handling of string types, specifically involving the Byte Order Mark (BOM), led to widespread out-of-bounds memory access issues. These vulnerabilities, which existed across all versions of Adobe Acrobat Reader DC since April 2015, enabled attackers to achieve significant security compromises, including information disclosure and, in some cases, remote code execution.
Liu's research transcends a simple bug report, offering a deep dive into the architectural flaws and development practices that underpin these pervasive security risks. He demonstrated how seemingly minor coding errors, such as an inadequate check for string encoding and the misuse of "security-enhanced" functions, could cascade into severe vulnerabilities. The presentation serves as a potent case study, emphasizing the importance of rigorous string handling, secure API usage, and the often-overlooked implications of legacy code in complex software environments.
Background
▶ Watch: Talk introduction and speaker background (0:00)
To comprehend the intricacies of the vulnerabilities presented, it's essential to first establish a foundational understanding of character sets, string types, and the mechanisms designed to handle them. The digital world relies on various character sets to represent text. Common examples include ASCII, a 7-bit encoding capable of representing 128 characters, and ANSI, an 8-bit encoding (one byte per character) that extends ASCII to 256 characters, often specific to a locale. More complex languages require broader character sets like UCS-2 (two bytes per character, supporting over 60,000 characters) and UTF-16 (two to four bytes per character, capable of representing over a million characters), which is based on UCS-2. While UTF-8 is also widely used, it was not the focus of this particular research.
Under the Windows development environment, strings are primarily categorized into ANSI strings and Unicode strings. An ANSI string is typically a char array, where each character occupies one byte, and its termination is signified by a single-byte null character (0x00). In contrast, a Unicode string (more precisely, a UCS-2 or UTF-16 string in this context) is represented by a wchar_t array, with each character occupying two bytes, and its termination requires a two-byte null character (0x00 0x00). This fundamental difference in character size and null terminator length is central to the vulnerabilities discussed.
Traditional string manipulation functions like strcpy, strncpy, and strlen are notorious for their susceptibility to buffer overflow attacks if not used with extreme care. To address these security concerns, Microsoft introduced "security-enhanced" versions, identifiable by an _s suffix (e.g., strcpy_s, wcsncpy_s). These functions typically include an additional parameter specifying the size of the destination buffer, aiming to prevent overflows. However, as Liu highlighted, even these enhanced functions can be rendered insecure if misused, for instance, by passing an overly large or incorrect buffer size, or if strncpy (without _s) fails to append a null terminator when the source string length exceeds the specified character count.
A critical concept for this research is the Byte Order Mark (BOM). For multi-byte character encodings like UTF-16, the order in which bytes are stored (e.g., least significant byte first - Little Endian (LE), or most significant byte first - Big Endian (BE)) can vary between systems. The BOM is a special Unicode character, U+FEFF, whose encoded form (either 0xFEFF for BE or 0xFFFE for LE) can be placed at the very beginning of a data stream to explicitly indicate its byte order. This 0xFFFE or 0xFEFF sequence at the start of a file or string is solely for specifying endianness, and its presence is meant to guide parsers in correctly interpreting the subsequent data. The speaker's research demonstrated how Adobe Reader's flawed interpretation and reliance on this two-byte sequence would lead to severe security implications.
Key Findings
▶ Watch: Adobe's string handling flaws and type confusion (16:00)
Ke Liu's extensive research into Adobe Acrobat and Reader uncovered a class of critical vulnerabilities rooted in two primary architectural and implementation flaws within Adobe's string handling mechanisms. These flaws, when combined, created a dangerous type confusion scenario, leading to out-of-bounds memory access.
The first fundamental flaw identified was Adobe's overly simplistic and insufficient method for determining string types. Adobe Reader implemented its own set of internal string functions, often with "safe" suffixes (e.g., string_n_len_safe), which acted as top-level APIs designed to automatically dispatch requests to either an ANSI or a Unicode implementation based on the detected string type. However, the logic for this type detection was severely flawed: it relied solely on checking if the first two bytes of a string matched the Byte Order Mark (BOM) sequence, 0xFEFF. Specifically, if (first_byte == 0xFE && second_byte == 0xFF), the string would be unconditionally treated as a Unicode string. This check is woefully inadequate because an ANSI string can legitimately begin with the byte sequence 0xFE 0xFF (representing two distinct ANSI characters, such as 'þ' and 'ÿ' in some code pages), without actually being a Unicode string.
The second critical flaw exacerbated the first: Adobe developers frequently misused the max_bytes_to_process or length parameters in their "security enhanced" string functions. Instead of passing the actual, safe buffer size, they often supplied a maximum possible value, suchs as 0x7FFFFFFF. By providing this arbitrarily large number, developers effectively disabled the very security features these functions were designed to provide. The functions, despite their "safe" designation, would then attempt to process an enormous amount of data, behaving much like their insecure, traditional counterparts (e.g., memcpy) without any bounds checking on the destination buffer.
The combination of these two flaws created a perfect storm for exploitation. When an ANSI string that happened to start with 0xFE 0xFF was passed to an Adobe API, the flawed type detection would misinterpret it as a Unicode string. Consequently, the string would be processed by Unicode-specific functions, which expect two bytes per character and a two-byte null terminator (0x00 0x00). However, the actual ANSI string was allocated only one byte per character and terminated with a single-byte null (0x00). The Unicode function, searching for its expected two-byte null terminator, would read past the allocated buffer, failing to find the 0x00 0x00 sequence. This led directly to out-of-bounds read vulnerabilities, allowing information disclosure, and in scenarios involving copying data, out-of-bounds write vulnerabilities, which could lead to heap corruption and ultimately code execution.
Ke Liu's methodology, which involved fuzzing possible input locations with malformed ANSI strings starting with 0xFE 0xFF, successfully uncovered approximately ten independent vulnerabilities. These vulnerabilities affected all versions of Adobe Acrobat Reader DC released since April 2015, highlighting the widespread impact of these fundamental design and implementation errors.
Technical Deep Dive
▶ Watch: CVE-2019-7032: Out-of-bounds read for info disclosure (20:30)
Adobe Reader's internal architecture for string handling played a pivotal role in the emergence of these vulnerabilities. The application implemented a set of custom string functions, designed as top-level APIs, which aimed to simplify string operations for developers. These APIs, such as string_n_len_safe and string_copy_safe, were intended to be "smart" – automatically detecting the string's encoding (ANSI or Unicode) and dispatching the request to the appropriate internal implementation.
Let's examine the behavior of one such function, string_n_len_safe, as illustrated in the presentation. This function typically accepts parameters like the string_to_be_processed, max_bytes_of_content_to_be_processed, and an error_handling_function_pointer. The critical decision point within this function, as highlighted by Liu, was an if statement: if (first_byte == 0xFE && second_byte == 0xFF). If this condition was met, the function would redirect the processing to its internal Unicode string handler; otherwise, it would default to the ANSI handler.
This simplistic check is the root of the type confusion. The 0xFE FF sequence, while serving as a Byte Order Mark for UTF-16, is not a definitive indicator of Unicode for all contexts. In an ANSI encoding, 0xFE and 0xFF are valid, distinct characters (e.g., 'þ' and 'ÿ' in Windows-1252). Therefore, an ANSI string legitimately starting with these bytes would be incorrectly identified as Unicode.
Compounding this error was the pervasive misuse of the max_bytes_of_content_to_be_processed parameter. Instead of providing the actual, safe buffer size for the destination, Adobe developers frequently supplied the value 0x7FFFFFFF. This large integer (equivalent to 2,147,483,647) effectively told the "safe" function to process an enormous number of bytes, far exceeding any reasonable buffer allocation. As a result, the security-enhancing aspect of these functions – their ability to prevent overflows by respecting a maximum length – was entirely negated. The functions effectively reverted to the behavior of unsafe primitives like memcpy, where the caller is solely responsible for ensuring sufficient buffer space.
Consider an ANSI string 0xFE 0xFF 0x2D 0x40 0x87 0x65 0x00. This string is 7 bytes long, with 0x00 acting as its single-byte null terminator.
- When passed to Adobe's
string_n_len_safe, the function observes0xFEand0xFFas the first two bytes. - The
ifcondition(first_byte == 0xFE && second_byte == 0xFF)evaluates to true. - The function then erroneously dispatches the string to the Unicode handler.
- The Unicode handler expects each character to be two bytes and anticipates a two-byte null terminator (
0x00 0x00). - It interprets
0xFE 0xFFas the BOM, then0x2D 0x40as the first character,0x87 0x65as the second character. - When it reaches the
0x00byte, it interprets it as the first byte of a two-byte character. It then continues reading past the allocated memory in search of the second byte of the character and eventually, a0x00 0x00null terminator.
This behavior directly leads to out-of-bounds read when calculating length or accessing data, and out-of-bounds write during copy operations. The underlying heap buffer, designed to hold a 1-byte-per-character ANSI string, is too small for the Unicode function's expectations, resulting in memory corruption. This fundamental disconnect between the perceived string type and its actual memory representation forms the bedrock of all the vulnerabilities discovered by Ke Liu.
Demo / Proof of Concept
▶ Watch: Advanced exploit techniques for code execution (31:30)
Ke Liu demonstrated the practical exploitability of these flaws through two distinct case studies, showcasing both information disclosure and code execution.
Case Study 1: CVE-2019-7032 - Out-of-Bounds Read (Information Disclosure)
This vulnerability was a classic out-of-bounds read, leading to the disclosure of sensitive memory contents. The proof-of-concept (POC) was remarkably simple, requiring just two lines of JavaScript embedded within a PDF document:
When Adobe Reader processed this JavaScript, the addField method created a field object. Subsequently, assigning a string starting with "FeFF" to the username property of this field triggered the vulnerability. As discussed in the technical deep dive, this ANSI string, starting with 0xFE 0xFF, was mistakenly treated as a Unicode string by Adobe's internal string handling functions. Since the underlying buffer for the username property was allocated for an ANSI string, the Unicode processing led to an out-of-bounds read, causing a crash that leaked memory contents. Liu found 18 different combinations (varying field types like text, radio, combo box, and properties like username, submitName, value) that could trigger this same underlying vulnerability.
Exploiting this for information disclosure involved a sophisticated heap manipulation strategy:
- Heap Spraying: The attacker would first spray the heap with numerous objects that contain virtual table pointers (vtables). These vtables are crucial for bypassing Address Space Layout Randomization (ASLR), as they point to code within loaded modules.
- Hole Creation: Half of these sprayed objects would then be freed, creating predictable "holes" in the heap memory.
- Triggering the Vulnerability: The out-of-bounds read vulnerability would then be triggered. By carefully orchestrating the heap layout, the OOB read would occur at a location where a leaked vtable pointer could be retrieved.
- ASLR Bypass: The leaked vtable pointer could then be used to calculate the base address of the
AcroForm.apimodule, effectively bypassing ASLR.
Liu highlighted an exploitation trick: standard JavaScript objects typically do not have vtable pointers. However, XFA attack objects, which can be created using the createNode function in XFA mode, do possess vtable pointers, making them ideal candidates. A challenge arose because the addField method used in the POC is not allowed within XFA mode. The workaround involved defining a static text field directly within the PDF code and then manipulating its properties from a JavaScript callback function (e.g., initialize event handler) to trigger the vulnerability. This allowed the attacker to successfully leak the base address of AcroForm.api.
Case Study 2: Out-of-Bounds Read/Write (Code Execution)
The second case study demonstrated a more severe out-of-bounds read/write vulnerability that could lead to full code execution. This vulnerability, fixed in October (no CVE number was provided in the talk), also stemmed from the same root cause: an ANSI string starting with 0xFE 0xFF being mishandled by Unicode functions, but in a context that allowed for controlled writes. The POC again involved simple JavaScript that, when executed, caused a heap corruption, indicated by an exception code related to a "corrupted heap block."
The exploitation strategy for code execution was more complex:
- Heap Spraying and Layout: The attacker would spray the heap with specific objects: a controllable string object (which would serve as the source for the OOB write) and multiple
ArrayBufferobjects. - Memory Hole Allocation: After freeing half of the objects, memory holes were created. The source string would be allocated in one hole, and a target
ArrayBufferobject in an adjacent hole. - Overwriting
byteLength: When thestring_copy_safefunction (which redirected to the Unicode handler) was triggered, it would write past the boundary of the source string, overwriting the adjacentArrayBuffer'sbyteLengthproperty. By changingbyteLengthto0xFFFFFFFF, theArrayBuffercould be tricked into believing it had access to a vast amount of memory, effectively achieving arbitrary read/write primitives.
Liu detailed crucial exploitation tricks for this stage:
DataViewPointer Corruption: ThebyteLengthvalue of anArrayBufferis often followed immediately by itsDataViewpointer in memory. A direct overwrite ofbyteLengthcould corrupt theDataViewpointer, causing an immediate crash.- Nulling the
DataViewPointer: To circumvent this, the attacker would intentionally set theDataViewpointer tonullwhen creating theArrayBuffer(using asmashfunction to duplicate objects). This way, overwritingbyteLengthto0xFFFFFFFFwould write the string terminator (0x00 0x00) at theDataViewpointer's address, which was already0x00, thus causing no side effect. - Absolute Memory Address Resolution: Even with arbitrary read/write relative to the
ArrayBuffer's backing store, achieving absolute memory read/write was still necessary for full control. This was accomplished by searching backward from theArrayBuffer's known address for specific patterns (e.g.,0xFF EE FF EE,0xF0 E0 D0 C0) that are characteristic of heap metadata or adjacent structures, allowing the calculation of the absolute memory address of the heap buffer.
Once arbitrary read/write at absolute memory addresses was achieved, the remaining steps for code execution (such as setting up Return-Oriented Programming (ROP) chains, bypassing Data Execution Prevention (DEP), and ASLR) are common techniques and were not elaborated upon in the talk. The demonstration concluded with the successful popping of a calculator, showcasing arbitrary code execution, albeit with sandbox protection temporarily disabled for clarity.
Liu also briefly mentioned other "trivial" cases of OOB reads he found, some triggered by utf-16 le specification rather than BOM, but which were not deemed exploitable for information disclosure or code execution. One notable case from 2016 involved PDF code (not JavaScript) specifying URI entry value as 0xFE FF, leading to an OOB read.
Defensive Implications
▶ Watch: Lessons learned and secure coding suggestions (36:30)
Ke Liu provided a thorough analysis of Adobe's patching efforts for the vulnerabilities he discovered, alongside his own recommendations for a more robust defense strategy.
For CVE-2019-7032 (the information disclosure out-of-bounds read), Adobe implemented a patch that involved appending three additional null characters (totaling four 0x00 bytes) at the end of the source string that was being processed. This change effectively ensured that even when the Unicode function mistakenly processed the ANSI string, it would encounter a sufficient number of null bytes within the intended bounds, thus preventing it from reading past the end of the allocated buffer. While not addressing the root cause of the type confusion, this pragmatic fix successfully mitigated the out-of-bounds read behavior.
For the out-of-bounds read/write vulnerability leading to code execution, Adobe issued a two-stage patch:
- First Patch: Similar to the previous fix, this patch involved adding two external null characters (totaling three
0x00bytes) to the end of the source stream. This had the effect of making thestring_copy_safefunction stop copying data earlier, limiting the potential overwrite to only a single byte. While an improvement, it didn't fully resolve the underlying issue and was still considered imperfect. - Second (Final) Patch: This more comprehensive fix directly addressed the problem at its source. When calculating the length of the source stream, Adobe transitioned from using an ANSI-centric length calculation to employing the
string_n_len_safefunction itself (which, in this context, would correctly handle the Unicode interpretation of the0xFE 0xFFstring). Crucially, it also allocated two extra bytes of memory to explicitly store the expected two-byte Unicode string terminator. By ensuring the destination buffer was adequately sized for the (misinterpreted) Unicode string, including its terminator, the out-of-bounds write was fully prevented.
Despite these successful patches, Liu emphasized that the underlying architectural flaws and incorrect usage patterns still persist in some parts of the codebase. He offered several key suggestions for developers and security architects:
- Distinguish String Types More Carefully: The simplistic
0xFE 0xFFcheck for Unicode strings is fundamentally flawed and insufficient. Robust parsing logic must be implemented to accurately identify character encodings. - Prioritize Unicode Internally: A more secure architectural approach would be to primarily use Unicode strings throughout the application's internal logic and only convert to ANSI strings when absolutely necessary (e.g., for compatibility with legacy APIs or external systems). Liu acknowledged that this is a significant undertaking, potentially affecting large portions of a codebase and facing developer resistance (humorously depicted with a meme of "developers don't want to touch the legacy code").
- Never Misuse "Security Enhanced" Parameters: The practice of passing
0x7FFFFFFF(or similar maximum values) as amax_bytesparameter to "security enhanced" string functions completely negates their protective features. Developers must always provide the actual, safe size of the destination buffer to ensure these functions perform their intended bounds checking. Neglecting this crucial detail renders "safe" functions as dangerous as their traditional counterparts. - Refactor Legacy Code: While challenging, refactoring outdated or insecure legacy code is essential for long-term security. Relying on quick fixes without addressing fundamental design flaws leaves systems vulnerable to new exploitation techniques.
- Increase Developer Security Consciousness: Ultimately, fostering a stronger security mindset among developers is paramount. Education and training on secure coding practices, especially regarding string handling, memory management, and API usage, can prevent similar vulnerabilities from being introduced in the future.
In summary, while Adobe's patches addressed the immediate vulnerabilities, Liu's presentation underscored the need for deeper architectural changes and a fundamental shift in development practices to truly secure complex applications like Adobe Reader.
Key Takeaways
- Flawed String Type Detection is Critical: Relying on simplistic checks like the
0xFE 0xFFByte Order Mark to determine string encoding is insufficient and can lead to dangerous type confusion, misinterpreting ANSI strings as Unicode. - "Security Enhanced" Functions Require Correct Usage: Functions designed for security (e.g.,
_sversions) are only effective if their parameters, especially maximum length arguments, are used correctly. Passing values like0x7FFFFFFFnegates their protective features and effectively reverts them to unsafe behavior. - Architectural Debt Leads to Vulnerabilities: Legacy code and architectural designs that don't account for modern security paradigms (like robust string handling) can become significant sources of widespread vulnerabilities, even in critical software.
- Sophisticated Exploitation Chains: Seemingly minor out-of-bounds reads can be chained with techniques like heap spraying, virtual table pointer leaks, and
ArrayBuffermanipulation to bypass ASLR and achieve arbitrary code execution. - Proactive Security is Key: Addressing fundamental design flaws, refactoring legacy code, and fostering security awareness among developers are crucial for preventing classes of vulnerabilities rather than merely patching individual instances.
About the Speaker(s)
Ke Liu is a Senior Security Researcher at Tencent Security Xuanwu Lab. He is a highly accomplished vulnerability researcher, having discovered and reported approximately 400 vulnerabilities across products from major technology companies including Adobe, Apple, Google, and Microsoft. His expertise has been recognized through his nomination for the prestigious Pwnie Awards in 2017. Ke Liu has previously shared his research on finding vulnerabilities in PDF readers at prominent conferences such as Black Hat Asia in 2017. He has also been consistently listed among the MSRC (Microsoft Security Response Center) Top 100 from 2016 to 2018. Furthermore, he demonstrated his offensive capabilities by successfully compromising Adobe Reader at the Tianfu Cup, a renowned hacking competition in China, and has been a speaker at Google's Escalade and Bugs SWAT events.
Reviews
Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT
Ke Liu drops a masterclass on how a two-byte parser confusion turns into arbitrary read/write in Adobe Acrobat. This isn't just a bug hunt; it's a structural takedown of Adobe's internal APIs and the developer habit of neutering safe functions. If you want to see how real memory corruption chains are built from logic flaws, watch this.
Heather Calloway (CISO) — SOLID
Ke Liu's research is a clinic in vulnerability discovery, detailing a four-year architectural flaw in Adobe Reader that enables remote code execution. While the exploit mechanics are for the offensive research community, the strategic takeaway for leadership is stark. It is a reminder of the massive, fragile attack surface living on every corporate endpoint through ubiquitous document handlers.