Extracting Info From Automotive Internet Units

Extracting Info From Automotive Internet Units

Speakers: Alexey Kondikov (Cyber Security Researcher, Navinfo Europe), Yuriy Serdyuk (Cyber Security Researcher, Navinfo Europe)

Conference: HITB Phuket

YouTube: https://www.youtube.com/watch?v=sRL-urIerKA

Overview

In their compelling presentation at HITB Phuket, Alexey Kondikov and Yuriy Serdyuk, cyber security researchers from Navinfo Europe, unveiled their meticulous methodology for extracting information from Automotive Internet Units (AIUs) found in modern vehicles. These AIUs, often overlooked yet critical components, serve as the car's primary interface with external networks, handling emergency calls, embedded SIM functionality, and over-the-air (OTA) updates. The researchers demonstrated how they overcame significant technical hurdles, including unique memory packaging, physical anti-tampering measures, and non-standard memory layouts, to successfully dump and analyze the firmware of these devices.

The talk highlights the critical security implications of these units, positioning them as "very juicy targets for exploitation" due to their deep integration with vehicle systems and constant network connectivity. By demonstrating a full chain of physical extraction, custom tool development, and reverse engineering of proprietary memory and file system formats, Kondikov and Serdyuk expose the vulnerabilities inherent in relying solely on physical obfuscation for automotive security. Their work provides invaluable insights for both automotive manufacturers to improve their security posture and for security researchers seeking to understand the attack surface of connected vehicles.

This detailed exploration into the inner workings of AIUs underscores a growing concern in the automotive industry: as cars become more connected, the attack surface expands dramatically. The ability to extract and analyze the firmware of these units can uncover critical vulnerabilities that could lead to remote compromise, data exfiltration, or even control over vehicle functions. The techniques presented are a testament to the persistence required in hardware security research and serve as a crucial warning about the need for robust, multi-layered security approaches in vehicle design.

Background

▶ Watch: Introduction to automotive internet access units (0:00)

Modern vehicles manufactured from approximately 2019 onwards are highly likely to incorporate some form of an Automotive Internet Unit (AIU). These units are typically found as dedicated PCBs "sandwiched" onto the main vehicle PCB, often encased in metallic shields. Despite varying car manufacturers or AIU vendors, the underlying architecture frequently converges on common designs. The core of these AIUs typically features a Qualcomm CPU, often from the MGM family, paired with a specialized memory chip.

The memory chips themselves present a significant challenge for security researchers. They are not standard eMMC memory, despite often sharing the same BGA162 packaging. Instead, they are unique packages containing both NAND Flash and DDR RAM within a single physical unit. This integration, coupled with the absence of readily available readers or sockets, immediately complicates data extraction. Furthermore, vendors often employ physical anti-tampering measures, such as gluing the chip to the PCB with a robust compound, making desoldering without damage virtually impossible. The internal memory layout is also highly customized, deviating from standard formats.

The researchers initially explored common methods for removing the stubborn glue compound. They attempted various polar solvents (like acetone and isopropanol) and non-polar solvents (like tetrachloroethylene), but these proved ineffective as the compound was not soluble. An ultrasonic bath was also tried, with various solvents, but this method was not only ineffective but also posed significant safety risks due to the heating and boiling of highly flammable and toxic liquids. Ultimately, a more destructive, yet effective, physical approach was required. This involved carefully cutting the section of the PCB containing the chip, then grinding the PCB layer by layer until only the compound remained on the memory chip. The remaining compound was then carefully cracked and removed using a high-temperature soldering iron. This delicate process, while destructive to the original board, was necessary to gain access to the raw memory chip.

Once the chip was isolated, the next challenge was reading it. Given the non-standard nature of the chip and the lack of commercial readers, the team had to design and build their own custom PCB connector board. After carefully reballing the chip (a common process for BGA packages), it was soldered onto this custom board. For the reader interface, they leveraged an FT2232 board, a versatile solution often used for custom hardware interfacing tasks. This bespoke setup allowed them to establish a connection to the NAND Flash memory, setting the stage for the raw data dump.

Key Findings

▶ Watch: Building a custom reader for non-standard memory (10:00)

The research yielded several critical discoveries, ranging from overcoming physical barriers to uncovering software flaws and proprietary data structures.

1. Custom Hardware & Data Extraction: The team successfully developed a custom PCB connector board utilizing an FT2232 chip to interface with the unique BGA162 NAND Flash + DDR memory package. This enabled them to dump the raw memory image despite the lack of commercial tools and the presence of physical anti-tampering glue.

1. Integer Overflow in nand2 Tool: During the initial data dumping using the open-source nand2 tool, the researchers observed anomalous behavior: binwalk revealed unaligned file system chunks and an entropy test showed two identical copies of data within the dump. This led to the discovery of an integer overflow vulnerability in nand2. The page_number argument, an integer (4 bytes), was being shifted by 16 bits in the address calculation. This overflow caused data loss and resulted in the tool reading the same memory region twice, producing a duplicated image. Fixing this bug in their modified nand2 implementation was crucial for obtaining a correct, non-duplicated memory dump.

1. Complex NAND OOB (Out-of-Band) Block Structure and ECC Discovery: Even after correcting the nand2 bug, attempts to mount extracted file systems (e.g., squashFS) failed, indicating corruption. The datasheet specified a page size of 4096 bytes data plus 224 bytes OOB, but observations showed varying OOB meta-information sizes (e.g., 6F or 4F bytes). Through careful analysis of plain text pages, they reverse-engineered the true page structure:

• The page was divided into chunks, each consisting of 214 bytes of data, followed by 1 byte (always FF), some data, and then F bytes of meta-information.
• They identified a 20-byte "unused" buffer in the last chunk, which contained residual data from previous operations rather than actual page data, a likely artifact of uninitialized buffers in the firmware.
• The final, corrected understanding of the page structure revealed 1000h bytes of actual data and E0h bytes of meta-information, aligning with the datasheet's total OOB size.
• The crucial meta field within the OOB block was identified as containing Error Correction Codes (ECC). By analyzing its high entropy and lack of monotonic increase, and then brute-forcing various well-known ECC algorithms and parameters, they successfully identified the specific ECC scheme used as BCH with a particular set of parameters. This allowed them to write a script to correctly fix data errors and remove the OOB blocks, yielding a clean, readable memory dump.

1. Discovery of Core Partitioning Scheme: Analysis of the cleaned memory dump's hex data revealed magic values (0x434F5245). Googling this led them to source code responsible for parsing core partitions. Using this insight, they developed a script to map out the entire disk structure, identifying various partitions including the Android kernel, Android operating system, and multiple UBI (Unsorted Block Image) partitions.

1. Proprietary "Good Archive" Format: Within the extracted partitions, they discovered files using a proprietary "good_arc" format, identified by a specific magic text. Although an existing Perl script (parse_cv.pl) failed to extract the content, it provided enough structural information to reverse-engineer the format. The good_arc structure was found to contain a header with the number of blocks, where each block stored pass and data items. The researchers then developed a Python script to successfully extract all files from these good_arc archives.

1. Consistency Across Different AIUs: The methodologies developed were validated on a second NAND Flash chip (JY990) from a different modern electrical car's AIU. Although this chip had different specifications (smaller page size of 2048 bytes, 64 bytes OOB, four chunks per page) and different BCH ECC parameters, the same systematic approach allowed them to successfully extract and analyze its memory dump, confirming the general applicability of their techniques.

Technical Deep Dive

▶ Watch: Discovering integer overflow bug in nand2 dumping tool (15:00)

The technical journey began with the physical extraction of the BGA162 packaged memory chip, which uniquely integrates NAND Flash and DDR RAM. The initial hurdle of the protective glue compound was overcome by first surgically removing the PCB section containing the chip. Then, a meticulous grinding process was employed, progressively removing layers of the PCB until only the chip and the compound remained. The compound itself was then carefully fractured using a high-temperature soldering iron, exposing the chip's pins.

With the chip liberated, the next step was to interface with it. As no off-the-shelf sockets or readers exist for this specific package, the researchers designed and fabricated a custom PCB connector board. This board provided the necessary physical interface for the reballed BGA162 chip. The logical interface was handled by an FT2232 board, a versatile USB-to-serial/JTAG/SPI/I2C converter, which allowed for low-level communication with the NAND Flash. This setup effectively transformed the unique memory chip into a readable device.

Initial attempts to dump the raw memory using the open-source nand2 tool revealed critical issues. An integer overflow was identified in the nand2 tool's calculation of the page_number to memory address. The page_number variable, defined as a 4-byte integer, was subjected to a bitwise left shift of 16. This operation, intended to calculate the physical address, caused an overflow for larger page numbers, effectively wrapping around the address space. As a result, the tool would read the same block of memory multiple times, leading to a dump where significant portions of data were duplicated, as evidenced by binwalk and entropy analysis. Correcting this integer overflow in their custom nand2 implementation was a fundamental step towards obtaining an accurate memory dump.

Even with a corrected nand2 tool, mounting file systems like squashFS from the dump consistently failed, indicating underlying data corruption. This pointed to an issue with how the nand2 tool or subsequent processing handled the Out-of-Band (OOB) data in the NAND Flash. The datasheet specified that each page consisted of 4096 bytes of main data and 224 bytes of OOB data. However, direct examination of the raw dump showed meta-information sections within the OOB block that were only 6F or 4F bytes long, not the full 224 bytes.

The researchers undertook a meticulous process of reverse-engineering the true page structure. By identifying pages containing known plain text, they could differentiate between actual data and meta-information. They discovered that a page was internally structured into multiple chunks. Each chunk was composed of 214 bytes of data, followed by a single byte (consistently FF), then some additional data, and finally F bytes of meta-information. A particularly insightful discovery was a 20-byte buffer in the last chunk, which often contained seemingly random data or remnants of previous operations, suggesting it was an "unused" buffer that was not properly cleared by the firmware. This highlighted a potential information leak or a coding oversight. After accounting for these internal chunk structures and the "unused" buffer, the team confirmed that the actual data payload was 1000h bytes, and the true meta-information (OOB) block was E0h bytes, finally matching the total 224 bytes specified in the datasheet.

Crucially, the meta field within the OOB block was identified as containing Error Correction Codes (ECC). The high entropy and non-monotonic nature of this field strongly suggested it wasn't a page counter or other sequential data. Through a systematic brute-force approach against a range of common ECC algorithms (e.g., Reed-Solomon, BCH) and their various parameter sets, they successfully identified the specific ECC algorithm as BCH with particular parameters. This breakthrough enabled them to develop a custom script that could not only remove the OOB blocks but also use the BCH ECC to correct any bit errors in the main data, yielding a perfectly clean and readable memory dump.

With a clean dump, the next phase involved understanding the disk's logical structure. Initial hex dump analysis of the corrected image revealed a distinctive magic value, 0x434F5245. A quick search for this value led them to open-source code related to parsing core partitions, which provided the necessary context to interpret the disk layout. They then wrote a script to parse this structure, revealing a detailed list of partitions, their offsets, and sizes. This included standard components like the Android kernel and Android operating system, alongside numerous UBI (Unsorted Block Image) partitions. For extracting data from the UBI partitions, they utilized UBI Reader, an open-source tool, successfully extracting various file systems, including squashFS images, certificates, and customer-specific partitions.

A notable discovery within the extracted file systems was a proprietary archive format, dubbed "good_arc," identified by its unique magic text. While an existing Perl script, parse_cv.pl, failed to fully extract the contents, it provided critical clues about the archive's internal structure. The good_arc format was found to contain a header specifying the number of blocks, with each block further containing pass and data items. Armed with this structural understanding, the researchers developed a Python script to fully decompress and extract all files from these good_arc archives, gaining access to even more layers of the AIU's software.

To validate their methodology, the entire process was replicated on a second AIU from a different modern car, which contained a NAND Flash chip marked JY990. This chip had different specifications: a smaller page size of 2048 bytes and an OOB block of 64 bytes, and its pages were structured into only four chunks. Furthermore, the ECC parameters for the BCH algorithm were different. Despite these variations, the systematic approach—from physical extraction and custom reader development to OOB analysis, ECC brute-forcing, and file system reverse engineering—proved universally applicable, demonstrating the robustness of their research framework.

Demo / Proof of Concept

▶ Watch: Extracting UBI and custom good_arc file systems (26:00)

While the talk did not feature a live, interactive demonstration in the traditional sense, the entire presentation served as a comprehensive proof of concept, illustrating each step of the successful information extraction process through detailed visuals and analytical results. The researchers meticulously documented their journey, providing concrete evidence at every stage.

The "demo" began with visual proof of the physical challenges: images of the metallic shields covering the PCB, the glued memory chip, and the various solvents and ultrasonic bath attempts that failed. The success of their physical extraction technique was demonstrated with photographs showing the PCB grinding process layer by layer, culminating in the isolated, cleaned memory chip. This was followed by images of their custom PCB connector board and the FT2232 setup, showcasing the bespoke hardware they engineered to interface with the unique BGA162 chip.

Crucially, the analytical "demo" was presented through a series of binwalk outputs and entropy plots. Initially, the binwalk results from the raw nand2 dump highlighted unaligned file system chunks and an entropy plot that clearly depicted two identical data segments, visually confirming the integer overflow issue. After fixing the nand2 bug and subsequently correcting the OOB blocks and applying BCH ECC, a dramatic improvement was shown: new binwalk outputs displayed perfectly aligned addresses and successful identification of numerous embedded file systems, demonstrating the integrity of the recovered data.

Further evidence of their work included hex dumps used to illustrate the 0x434F5245 magic value for core partitions and the "good_arc" magic text, which were critical for understanding the disk and archive structures. The output from their custom scripts, such as the list of partitions with their offsets and sizes, provided a clear map of the AIU's storage. Finally, the successful extraction of various file systems using UBI Reader and their custom Python script for good_arc archives was demonstrated by presenting lists of extracted files and folders, showcasing the rich data they were able to recover. This comprehensive visual and analytical evidence throughout the presentation effectively served as a robust proof of concept for their entire methodology.

Defensive Implications

▶ Watch: Summary of research and future security assessments (30:00)

The research presented by Kondikov and Serdyuk carries significant defensive implications for the automotive industry and connected device manufacturers. The findings underscore that relying solely on physical obfuscation and non-standard hardware for security is a fundamentally flawed strategy against determined adversaries.

1. Inadequacy of Physical Obfuscation: The talk clearly demonstrates that physical barriers like metallic shields, specialized BGA packaging, and even strong glue compounds are insufficient security measures. With sufficient motivation and resources, these physical protections can be bypassed, exposing the underlying hardware and data. Manufacturers must understand that physical security is only one layer and cannot be the primary defense.

1. Importance of Robust Data Encryption: The ease with which raw memory dumps were obtained, even from non-standard chips, highlights the critical need for strong encryption of all sensitive data at rest on these automotive units. Firmware, proprietary algorithms, configuration files, user data, and especially any cryptographic keys must be encrypted with robust, modern algorithms. Ideally, encryption keys should be stored in a Hardware Security Module (HSM) or a Trusted Execution Environment (TEE) that is resistant to physical extraction.

1. Secure Boot and Integrity Checks: The presence of squashFS with integrity checks is a positive sign, but the ability to extract and manipulate the firmware means that secure boot mechanisms must be robustly implemented and verified. This includes cryptographic signatures for all firmware components, ensuring that only trusted and unaltered code can execute. Any modification to the boot chain or core software should prevent the device from operating or trigger an alert.

1. Secure Coding Practices for Low-Level Tools: The discovery of an integer overflow in the nand2 tool, an apparently simple utility, serves as a stark reminder that even fundamental tools and low-level drivers can harbor critical vulnerabilities. Automotive firmware development must adhere to rigorous secure coding standards, including comprehensive input validation and bounds checking, to prevent similar logic flaws that could be exploited by an attacker.

1. Protection of Proprietary Information and Trade Secrets: The extraction of proprietary file system formats (like "good_arc") and the ability to analyze the full firmware expose manufacturers' intellectual property. This includes not just code but also design decisions, algorithms, and potentially configuration details that could aid further attacks. Robust obfuscation techniques for critical code sections and sensitive data within the firmware should be considered in addition to encryption.

1. Supply Chain Security: The presence of specific Qualcomm CPUs and memory chip families suggests common components across different vendors. This highlights the importance of supply chain security. Manufacturers must audit their third-party components and ensure that security vulnerabilities are not introduced at the hardware or firmware level before integration into the vehicle.

1. Threat Modeling and Continuous Assessment: Automotive manufacturers need to adopt comprehensive threat modeling that includes physical attacks and deep-dive firmware analysis. Regular and thorough security assessments, including hardware penetration testing, are essential to identify and remediate vulnerabilities before they are exploited in the wild.

In essence, the talk serves as a wake-up call, demonstrating that physical access to an AIU can lead to a full compromise if robust software and cryptographic protections are not in place. Defenders must shift their focus from merely deterring physical access to designing systems that remain secure even when physical access is achieved.

Key Takeaways

• Physical Obfuscation is Not Sufficient: Relying on glue, custom packaging (BGA162 NAND+DDR), or metallic shields provides only a speed bump, not true security, against determined attackers.
• Custom Hardware Requires Custom Tooling: Interfacing with non-standard memory chips often necessitates building bespoke hardware (e.g., custom PCB connector with FT2232) and developing specialized software to dump raw data.
• Open-Source Tools Can Have Critical Flaws: Even widely used low-level tools like nand2 can contain significant bugs (e.g., integer overflow in page address calculation) that lead to incorrect data dumps, requiring deep analysis and correction.
• Deep Understanding of Memory Architecture is Crucial: Successfully extracting data from NAND Flash requires meticulous reverse-engineering of page structures, OOB block layouts, and the specific Error Correction Codes (ECC) (e.g., BCH) used, as vendors often deviate from standard patterns.
• Automotive Firmware is a Rich Target: Extracted firmware contains a wealth of information, including proprietary file system formats (e.g., "good_arc"), operating system components (Android kernel, UBI partitions), certificates, and potentially sensitive configuration data, offering a vast attack surface for further exploitation.
• Robust Cryptographic Protections are Essential: Given the demonstrated ease of physical data extraction, strong encryption of firmware and sensitive data at rest, coupled with secure boot and hardware-backed key storage (HSM/TEE), is paramount for the security of Automotive Internet Units.

About the Speaker(s)

Alexey Kondikov and Yuriy Serdyuk are experienced Cyber Security Researchers at Navinfo Europe, a company based in the Netherlands. Their work focuses on conducting cyber security research and penetration tests specifically for modern cars and various automotive devices. Their expertise lies in dissecting the complex security architectures of connected vehicles, uncovering vulnerabilities, and developing methodologies for deep-level analysis, as powerfully demonstrated in their talk on extracting information from Automotive Internet Units. Yuriy Serdyuk holds the title of Lead Security Researcher at Navinfo Europe, indicating his senior role in guiding their research efforts.