Biometrics System Hacking in the Age of the Smart Vehicle
Huajiang Chen (Senior Security Researcher), Li Siwei (Security Researcher)
Car Hacking Village @ DEF CON 30 · Day 1 · Car Hacking Village
Overview
In an increasingly connected world, smart vehicles are rapidly integrating advanced technologies, including sophisticated biometric authentication systems, for user access and control. This talk, "Biometrics System Hacking in the Age of the Smart Vehicle," presented by Huajiang Chen ("Kevin2600") and Li Siwei, delves into the vulnerabilities inherent in these systems. The researchers systematically explore various attack vectors against both facial and speaker recognition systems, demonstrating how these can be bypassed to gain unauthorized access to vehicles and other smart devices.

Key moments
- 0:00 Introduction to Biometric Authentication in Vehicles
- 6:50 Demo: Replay Attack on Tesla via Siri
- 10:50 Demo: Speech Synthesis Spoofing Smart Speakers
- 20:30 Demo: Xiaomi Smart Lock Facial Spoofing
- 27:50 Demo: Adversarial Mask Bypasses Car Face ID
- 31:00 Modifying Face ID Threshold with Root Access
- 33:50 RF Replay Attack on Access Control System
Biometrics System Hacking in the Age of the Smart Vehicle
Speakers: Huajiang Chen (Senior Security Researcher); Li Siwei (Security Researcher)
Conference: Car Hacking Village @ DEF CON 30
YouTube: https://www.youtube.com/watch?v=t28rgINaaZk
Overview
In an increasingly connected world, smart vehicles are rapidly integrating advanced technologies, including sophisticated biometric authentication systems, for user access and control. This talk, "Biometrics System Hacking in the Age of the Smart Vehicle," presented by Huajiang Chen ("Kevin2600") and Li Siwei, delves into the vulnerabilities inherent in these systems. The researchers systematically explore various attack vectors against both facial and speaker recognition systems, demonstrating how these can be bypassed to gain unauthorized access to vehicles and other smart devices.
The presentation highlights the critical security implications of relying on biometrics, especially when integrated into complex automotive environments. As car manufacturers move away from traditional key fobs towards more convenient, but potentially less secure, authentication methods like facial and voice recognition, understanding these attack surfaces becomes paramount. Chen and Siwei provide a comprehensive analysis of spoofing techniques, ranging from simple replay attacks to advanced adversarial machine learning examples, showcasing real-world exploits against various smart devices and, crucially, vehicles.
This research is vital for security professionals, automotive engineers, and consumers alike, shedding light on the evolving threat landscape of smart vehicles. It underscores the necessity for robust anti-spoofing mechanisms, secure system architectures, and rigorous penetration testing to safeguard against unauthorized access and ensure the integrity of vehicle control systems in an era where "something you are" is becoming a primary means of authentication.
Background
▶ Watch: Introduction to Biometric Authentication in Vehicles (0:00)
The evolution of authentication in smart vehicles has seen a significant shift towards multi-factor authentication (MFA), incorporating elements like "something you have," "something you know," and increasingly, "something you are." Traditionally, "something you have" was a physical key fob, which has gradually been replaced or augmented by NFC cards and mobile phone apps, as seen with Tesla's early reliance on NFC and Bluetooth Low Energy (BLE). "Something you know" has manifested in features like Ford's door unlock PIN codes. The most recent and rapidly expanding frontier is "something you are," leveraging biometrics such as fingerprint readers, face ID, and voice recognition for vehicle access and control. Companies like BMW and Honda are integrating voice prints with smart speakers to enable car control, while others are using face ID for driver profiles and authentication.
The development of speaker recognition dates back to the 1960s, evolving from simple template matching to more complex probability models like Gaussian Mixture Models (GMM) and feature classification using i-vectors and x-vectors. Modern speaker recognition systems typically involve feature extraction and feature matching, comparing extracted voice features against a database of registered users. These systems can be categorized by the content they recognize: fixed vocabulary recognition (e.g., "Hey Siri"), fixed voice plus random content recognition (e.g., "Hey Bob 2600" where "2600" is random), and random content recognition where the entire phrase is generated randomly.
Similarly, facial recognition technology has progressed from low-dimension characterization to sophisticated 3D face recognition. Its application in vehicles includes driver monitoring systems, driver profile management, and direct owner authentication. The facial recognition procedure typically involves four stages: face detection (identifying a human face), liveness detection or face anti-spoofing (determining if the face is real or a spoof), feature extraction (identifying unique facial characteristics), and feature matching (comparing extracted features with a database). Liveness detection is crucial, often requiring user actions like opening the mouth or shaking the head, or employing silent methods based on skin texture and motion frequency. The increasing reliance on these biometric modalities, without adequate security considerations, creates fertile ground for sophisticated cyberattacks.
Key Findings
▶ Watch: Demo: Speech Synthesis Spoofing Smart Speakers (10:50)
The research by Chen and Siwei uncovered several critical vulnerabilities and effective attack methodologies against biometric systems in smart vehicles and related smart devices. A primary finding is that most biometric systems, despite their perceived sophistication, are susceptible to various forms of spoofing attacks, ranging from remarkably simple to highly advanced.
For speaker recognition, even basic replay attacks using recorded audio proved effective against systems with fixed vocabularies or lacking robust anti-spoofing. More concerning was the success of speech synthesis (TTS) attacks using advanced models like Google's Tacotron 2, which could impersonate voices to bypass authentication and even initiate online purchases. The researchers also demonstrated the efficacy of adversarial example attacks on voice recognition, where imperceptible perturbations to audio data could cause machine learning models (e.g., DeepSpeech) to misclassify speech, effectively spoofing a genuine speaker.
On the facial recognition front, the findings were equally stark. Simple photo attacks and even the "Mr. Orange attack" (drawing a face on an orange) successfully bypassed basic face detection in systems lacking proper liveness detection. For systems with liveness detection, physical bypasses like cutting holes in photos or using custom face sculptures proved effective. The most advanced facial recognition attacks involved adversarial example masks, which, by introducing calculated perturbations, could deceive advanced face ID systems in mobile phones and, critically, smart vehicles, enabling engine start and driving off.
Perhaps the most significant finding related to vehicle security was the potential for root access on the vehicle's Infotainment/In-Vehicle Infotainment (IVI) system. Gaining root privileges allowed direct manipulation of biometric system configurations, specifically the threshold value for authentication and the ability to replace stored biometric features. This demonstrated that even if the biometric algorithms themselves are robust, an insecure underlying operating system can completely undermine their security. Finally, the research highlighted that even if biometric systems are strong, an attacker might bypass them by exploiting alternative access control mechanisms, such as simple RF replay attacks on remote controllers, effectively sidestepping the biometric layer entirely.
Technical Deep Dive
▶ Watch: Demo: Xiaomi Smart Lock Facial Spoofing (20:30)
The technical depth of the presented attacks spans both speaker recognition and facial recognition systems, demonstrating a breadth of techniques from basic physical bypasses to advanced machine learning manipulation.
For speaker recognition spoofing, the speakers detailed several methods:
- Impersonation Attack: An attacker attempts to mimic the victim's voice. While conceptually simple, this is often easily detected by modern speaker recognition systems.
- Replay Attack: This is a more effective method, particularly against systems that use fixed sentence verification. The attacker records a victim speaking a required phrase (e.g., "Hey Siri") and then replays the recording during authentication. The talk demonstrated this by using a recorded voice to control a Tesla app via Siri, successfully opening the car's charge port, and in one instance, unlocking the car door through an app that didn't require extra authentication. Spectrogram analysis showed high similarity between the replayed and genuine voice, explaining its effectiveness against systems without voice anticipation or advanced anti-spoofing.
- Speech Synthesis (TTS) Attack: This leverages text-to-speech models to generate arbitrary speech in a target's voice. The researchers used Google's Tacotron 2, an open-source model, which comprises three main parts:
- Speaker Encoder Module: Extracts speaker features from a short audio sample.
- Synthesizer Module: Combines the speaker features with text to generate a mel spectrogram, which integrates the speaker's voice characteristics into the specified text.
- Vocoder Module: Generates the final audio waveform from the mel spectrogram.
The team demonstrated this by spoofing a Baidu smart speaker (even with anti-spoofing enabled) and a T-mode speaker. For the T-mode speaker, which used a "fixed vocabulary plus random content" model, they used a "monkey bird" technique to generate the random numbers required, ultimately leading to a successful online purchase.
- Adversarial Example Attack (Voice): This involves adding calculated perturbations (small, often imperceptible changes) to speech data to intentionally mislead a machine learning model. The talk showcased this against an open-source project called DeepSpeech. A genuine recording was correctly recognized, but a specially crafted recording with adversarial perturbations was translated differently, indicating a successful attack where the system was deceived.
For facial recognition spoofing, the attack vectors were categorized by the stage of the recognition process they targeted:
- Face Detection Attacks:
- Photo Attack: Printing a victim's photo on paper. Demonstrated successfully against a Xiaomi smart lock.
- Mr. Orange Attack: Drawing a human face on an orange. This surprisingly simple method fooled the Xiaomi smart lock into detecting a "human" face, even estimating its age (e.g., 24 years old), highlighting a lack of basic liveness detection.
- Liveness Detection / Anti-Spoofing Bypass Attacks:
- Cut-Hole Attack: For systems requiring actions like opening the mouth or shaking the head for liveness detection, attackers can cut holes in the eye or mouth areas of a photo to perform the required actions, bypassing detection.
- Face Sculpture Attack: This involves using a customized 3D face sculpture, which can bypass systems that lack robust human skin texture detection. A video demo showed this successfully bypassing the face ID on a Huawei P30 Pro.
- Adversarial Example Attack (Face): Similar to voice, this involves adding small, calculated perturbations to an image to trick the machine learning model. These perturbations are often incorporated into a specially designed mask. The research, referencing work from Tsinghua University, indicated that perturbing areas around the eyes and nose yield higher success rates. The attacks were demonstrated as black-box adversarial examples, where the attacker doesn't have full knowledge of the target model but uses techniques like ensemble training to improve transferability. Demos included bypassing face ID on a Huawei P30 Pro, a Wema car, and an undisclosed car, where the attack allowed the engine to be turned on and driven off.
- Feature Replace Attack: This specific attack involves putting a target face image onto the attacker's face, partially covering it. This method was shown to deceive some 2D facial recognition systems with a high success rate, as demonstrated on a Xiaomi Note 9.
- Feature Matching Attacks (requiring root access): These are particularly insidious as they target the core configuration and stored data of the biometric system, often requiring prior access to the vehicle's Infotainment/In-Vehicle Infotainment (IVI) system.
- Threshold Value Attack: On an Android-based IVI system (from an undisclosed car) where the researchers obtained root access via ADB, they found that the threshold value configuration file for facial recognition was stored locally. By modifying this value to a very low level and rebooting the IVI, they could bypass the "tight" anti-spoofing that previously prevented all other spoofing methods.
- Feature Value Attack: In another penetration test scenario, the team gained root access to a system with two human features stored in its database. They were able to replace one of these features with their own, allowing both the attacker and the original victim to log in successfully, effectively creating a stealthy backdoor.
- Extra Function Attack (RF Replay): As a final, pragmatic point, the speakers noted that even if biometric systems are perfectly secure, many access control systems (including those for vehicles) have multiple features enabled. They demonstrated a successful RF replay attack on a smart lock that could also be opened by a fixed-code radio remote controller, highlighting that "it's not a bug, it's a feature" – meaning, a less secure fallback mechanism can negate the security of more advanced systems.
The comprehensive summary provided at the end of the talk categorized these techniques: Mr. Orange, face photo, and face sculpture attacks for face detection; feature replace and adversarial example attacks for face anti-spoofing; and feature value and threshold value attacks for feature matching, alongside the RF replay for extra functions.
Demo / Proof of Concept
▶ Watch: Modifying Face ID Threshold with Root Access (31:00)
The talk was rich with practical demonstrations and proof-of-concept videos, showcasing the real-world applicability of the discussed attack techniques across various smart devices and, crucially, vehicles.
For speaker recognition spoofing, the initial demo highlighted a replay attack against Siri. Using a recorded voice command, the researchers successfully interacted with a third-party app controlling a Tesla, demonstrating the ability to open the car's charge port, even when the car door remained locked. This underscored the vulnerability of voice assistants when integrated into vehicle control, especially when associated apps lack robust multi-factor authentication. A more advanced demonstration involved speech synthesis (TTS) attacks using Google's Tacotron 2. This was first shown against a Baidu smart speaker, successfully spoofing a registered voice despite the speaker's anti-spoofing capabilities. A subsequent demo targeted a T-mode speaker used for online purchases. Here, the attackers used a "monkey bird" method to generate the random content required by the system, ultimately enabling them to impersonate a user and initiate a purchase, showcasing a direct financial impact. The adversarial example attack on voice was demonstrated against the DeepSpeech project, where a crafted audio recording, imperceptibly modified, caused the system to misinterpret the speech, confirming the efficacy of perturbation-based attacks.
The facial recognition spoofing demos were equally compelling. Early demonstrations targeted a Xiaomi smart lock. The "Mr. Orange attack," where a human face was drawn on an orange, surprisingly fooled the lock's face detection, even estimating the "age" of the orange. This was followed by successful face photo attacks and adversarial example mask attacks against the same lock, where a photo or a specially designed mask was detected as the legitimate user. For systems with more advanced liveness detection, the researchers demonstrated a face sculpture attack against a Huawei P30 Pro, where a custom-made 3D sculpture of a face successfully bypassed the phone's face ID.
The most critical demonstrations involved adversarial example masks against vehicle systems. Videos showed an attacker wearing a modified mask successfully bypassing the face ID login system on a Huawei P30 Pro. This was extended to automotive targets, with a demonstration against a Wema car's welcome face ID system, which was successfully bypassed using a similar modified mask. The most alarming demo, though the car's name was undisclosed, showed an attacker using an adversarial example mask to bypass the vehicle's face ID system, allowing them to turn on the engine and drive off. This vividly illustrated the severe security implications for smart vehicles. Another facial recognition attack, the feature replace attack, was demonstrated on a Xiaomi Note 9, where a special mask designed to incorporate target facial features enabled consistent bypass.
Finally, the talk showcased attacks that leveraged root access to the vehicle's IVI system. In a demo involving an undisclosed car, the researchers demonstrated the threshold value attack. Initially, the car's face ID system, with its "tight" anti-spoofing, resisted all spoofing attempts. However, after gaining root access, modifying the locally stored threshold value configuration file, and rebooting the IVI system, the face ID was successfully bypassed, allowing login. The talk concluded with a reminder of simpler, non-biometric bypasses, demonstrating an RF replay attack that successfully opened a smart lock, emphasizing that multiple access points can lead to vulnerabilities.
Defensive Implications
▶ Watch: RF Replay Attack on Access Control System (33:50)
The detailed findings from this research present a clear mandate for robust defensive strategies in the design and implementation of biometric systems, particularly within the automotive sector. Defenders must adopt a multi-layered approach to counteract the sophisticated spoofing techniques demonstrated.
Firstly, liveness detection and anti-spoofing mechanisms for both facial and speaker recognition must be significantly enhanced. For facial recognition, this means moving beyond simple 2D analysis to incorporate 3D sensing, infrared cameras (as demonstrated by Ford's system, which saw nothing from a 2D photo), multi-frequency motion analysis, and skin texture analysis that is resilient to advanced physical spoofs like custom sculptures. For speaker recognition, robust voice anticipation, dynamic challenges (e.g., asking the user to repeat randomly generated phrases, as seen with "monkey bird" attacks), and multi-modal verification (combining voice with another factor) are essential to defeat replay and speech synthesis attacks. The system should be able to detect unnatural speech patterns or digital artifacts.
Secondly, the security of the Infotainment/In-Vehicle Infotainment (IVI) system is paramount. Manufacturers must implement stringent measures to prevent root access or unauthorized modification of critical system files. This includes:
- Strong access controls: Limiting ADB debugging capabilities in production vehicles.
- Secure boot mechanisms: Ensuring the integrity of the operating system and applications.
- Encryption of sensitive data: Biometric feature databases and configuration files (like threshold values) must be encrypted and protected against local modification.
- Integrity checks: Regularly verifying the integrity of critical system files to detect tampering.
- Principle of Least Privilege: Limiting the permissions of applications and services running on the IVI.
Thirdly, the integration of biometrics for critical vehicle functions, such as engine start, demands extreme caution. Biometric authentication should always be part of a robust multi-factor authentication (MFA) scheme, not a standalone solution. For example, requiring a biometric scan in addition to a physical key fob or a PIN code would significantly raise the bar for attackers. The convenience of biometrics should never come at the expense of security for life-critical functions.
Furthermore, automotive manufacturers must conduct rigorous security audits and penetration testing that specifically target AI/ML models, including testing against adversarial examples. This requires specialized expertise to identify and mitigate vulnerabilities that arise from the inherent characteristics of machine learning algorithms. The "black box" nature of some attacks means that even without knowledge of the internal model, attackers can find ways to trick the system.
Finally, all alternative access control mechanisms must be secured to the same standard as the primary biometric system. The demonstration of a simple RF replay attack bypassing a smart lock highlights that a single weak link can compromise the entire security chain. This means ensuring that all key fobs, NFC cards, mobile apps, and remote controllers use strong, rolling-code encryption and are protected against replay and other common attacks. The "not a bug, it's a feature" mindset regarding less secure fallback options must be eliminated.
Key Takeaways
- Biometrics introduce significant new attack surfaces for smart vehicles: As cars increasingly adopt facial and speaker recognition for access and control, they become targets for various spoofing techniques.
- Both simple and advanced spoofing methods are effective: Basic replay attacks for voice and photo/sculpture attacks for face can bypass systems lacking robust anti-spoofing, while advanced AI-driven methods like speech synthesis and adversarial examples can deceive more sophisticated models.
- Root access to IVI systems is a critical vulnerability: Gaining root privileges on a vehicle's infotainment system allows attackers to directly modify biometric configuration files (e.g., authentication thresholds) or replace stored biometric features, completely undermining the system's security.
- Robust liveness detection and anti-spoofing are essential but often insufficient: While crucial, even advanced liveness detection can be bypassed by physical manipulations (e.g., custom sculptures) or sophisticated adversarial examples, necessitating continuous research and development in this area.
- Multi-factor authentication and secure fallback mechanisms are non-negotiable: Biometrics should be combined with other authentication factors, and all alternative access methods (e.g., RF remotes) must be secured to the same high standard to prevent attackers from exploiting the weakest link.
- The evolving nature of AI/ML attacks demands continuous vigilance: As biometric systems become more reliant on machine learning, they become susceptible to adversarial examples, requiring ongoing security audits and research into new defensive strategies for AI models.
About the Speaker(s)
Huajiang Chen ("Kevin2600") and Li Siwei are distinguished security researchers. Kevin2600 holds the title of Senior Security Researcher, while Li Siwei is a Security Researcher. Together, they have conducted extensive research into vehicle-related vulnerabilities, uncovering and presenting numerous interesting bugs at prominent security conferences, including DEF CON, over several years. Their work focuses on the intersection of automotive technology and cybersecurity, particularly exploring emerging attack surfaces in smart vehicles.
Reviews
Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT
Kevin2600 and Li Siwei show up with actual receipts on vehicle biometrics, bypassing face and voice auth on modern smart cars. They mix standard adversarial ML tricks with brutally practical IVI local-root exploits to show how vendors bolt high-tech biometrics onto fundamentally broken architectures.
Heather Calloway (CISO) — MUST SEE
This is a precise technical demonstration of biometric authentication failures in modern smart vehicles, covering everything from adversarial ML masks to local OS compromise. It proves that vehicle biometrics cannot currently serve as a high-assurance security boundary for critical actions. It is highly relevant for product security architecture teams and leaders managing executive vehicle fleets.