The Heat is On: Understanding and Mitigating Vulnerabilities of Thermal Image Perception in Autonomous Systems

The Heat is On: Understanding and Mitigating Vulnerabilities of Thermal Image Perception in Autonomous Systems

Speakers: Sri Hrushikesh Varma Bhupathiraju

Conference: NDSS Symposium 2026

YouTube: https://www.youtube.com/watch?v=QoQCE99bjWo

Overview

Thermal cameras are increasingly integrated into autonomous systems -- robot taxis (Waymo, Nuro), robotic platforms (RAS), and drones (DJI, Skydio) -- to enhance perception in low-light conditions and adverse weather. This talk presents the first security analysis of thermal camera signal processing, uncovering three novel vulnerabilities in the image acquisition, calibration, and equalization pipelines that can be exploited using nothing more than a $20 reptile heating lamp and aluminum foil. The attacks achieve 100% missed detection of genuine obstacles and 91% fake obstacle detection in real-world driving scenarios at up to 40 km/h, tested on three commercial thermal cameras: FLIR Boson (40% market share), InfiRay T2S, and FPV XKC130. Critically, the calibration attack produces delayed artifacts -- fake obstacles that appear at a controlled time after the attack has terminated, making it particularly stealthy. The researchers also design signal processing mitigations that address all three vulnerabilities simultaneously, accounting for their interdependence.

Background

▶ Watch: Three vulnerabilities in thermal camera signal processing pipeline (1:00)

Thermal cameras capture infrared radiation to build heat maps of the surrounding environment, enabling perception beyond visible light. Unlike RGB cameras, thermal cameras have unique physical properties: high dynamic range requiring periodic calibration, specialized germanium lenses, and a signal processing pipeline consisting of image acquisition, calibration (removing thermal drift noise), and equalization (improving contrast for obstacle detection).

Previous security research on camera-based perception has focused on RGB cameras, discovering vulnerabilities in lensing, image acquisition, and hardware. However, these vulnerabilities do not directly transfer to thermal cameras due to their distinct sensor characteristics and specialized signal processing.

Thermal cameras are used for obstacle detection in autonomous systems, with DNN-based detection models and sensor fusion (combining RGB and thermal) applied to the processed thermal images. The cameras tested represent automotive (FLIR Boson), robotics (InfiRay T2S), and drone (FPV XKC130) applications.

Key Findings

▶ Watch: Calibration attack: delayed artifacts appear after heat source removal (4:30)

Three distinct vulnerabilities in thermal signal processing:

1. Ghost artifacts (image acquisition): Bright heat sources create secondary images (ghosts) through lens reflections, similar to lens flare in RGB cameras. Unlike RGB ghosts that take the aperture shape, thermal camera ghosts preserve the structure of the heat source due to circular apertures and specialized materials. An attacker can shape a heat source (using aluminum foil) to create ghost artifacts resembling pedestrians or obstacles. An 80C heat source achieves >90% attack success rate regardless of artifact pose.

1. Calibration artifacts (delayed attack): Thermal cameras periodically calibrate to remove thermal drift noise. Heat sources cause overcompensation of the calibration offset, creating dark artifacts that persist and intensify over time even after the heat source is removed. A 30-second exposure triggers >90% success rate for fake obstacle detection. The artifact appears 10 seconds to 2+ minutes after the attack terminates, making it the most stealthy variant.

1. Equalization artifacts (missed detection): Equalization algorithms redistribute pixel intensities for better contrast. Strategic heat sources trigger linear behavior in equalization, causing information loss that reduces pixel intensity of genuine obstacles, leading to missed detection. Detection performance drops linearly with temperature, and pedestrians are most affected (their higher expected temperatures make intensity reduction more impactful).

Vulnerabilities are interdependent: The ghost artifact intensity depends on the equalization algorithm; the calibration attack strength is proportional to equalization-induced pixel changes. Individually fixing one vulnerability can make others stronger, requiring simultaneous mitigation.

Technical Deep Dive

▶ Watch: >90% attack success rate with 80C heat source and 30-second exposure (7:30)

The attack uses a standard reptile heating lamp (~$20) as the heat source, making it highly accessible. The infrared radiation is invisible, adding stealth. Aluminum foil shapes the heat source to create arbitrary-shaped artifacts (human/pedestrian silhouettes) for ghost and calibration attacks.

Ghost attack mechanism: In thermal lenses, bright heat sources produce reflections that form secondary images at pixel-opposite coordinates from the original source. Because thermal camera apertures are circular and lens materials preserve structural information, the ghost retains the shape of the heat source rather than becoming an aperture-shaped blob.

Calibration attack mechanism: Thermal cameras use bolometers that heat up during operation, requiring periodic offset calibration. The calibration algorithm calculates an offset by averaging recent frames. When a heat source is present during calibration, it biases the offset. After the source is removed, the over-corrected offset creates a dark artifact in the region where the source was. Due to the averaging nature, the artifact intensifies over time (gets darker frame by frame), and the attacker can control the delay and intensity by adjusting temperature, exposure duration, and timing relative to calibration cycles.

Equalization attack mechanism: Thermal cameras apply histogram equalization (e.g., CLAHE/plateau equalization) to improve contrast. Heat sources create spikes in the image histogram, triggering linear equalization behavior that compresses the dynamic range of the rest of the image, reducing contrast of genuine obstacles below detection thresholds.

Mitigations designed for interdependence: Ghost mitigation suppresses pixels at opposite coordinates of heat sources >80C (100% real-world success). Calibration mitigation limits the offset to a calculated threshold based on camera-specific attack requirements (100% artifact suppression, no degradation of genuine detection). Equalization mitigation clips heat source histogram spikes before redistribution, improving detection from 0% to 96%.

Demo / Proof of Concept

▶ Watch: Vulnerability interdependence: fixing one can strengthen others (9:30)

Real-world testing used scenarios with a vehicle moving at 40 km/h approaching a pedestrian from 50 meters. The equalization attack caused complete missed detection of the pedestrian. The calibration attack triggered fake obstacle detection 10 seconds to 2 minutes after the heat source was removed, with detection persisting across all three camera models. The ghost attack achieved 76% success rate on the FLIR Boson at 2.5 km/h (limited by heat source size at greater distances). All mitigations were validated in real-world conditions. Vulnerabilities were disclosed to camera vendors.

Defensive Implications

▶ Watch: Real-world demo: equalization attack at 40 km/h causes missed detection (12:00)

For autonomous vehicle and drone manufacturers integrating thermal cameras for obstacle detection, these vulnerabilities represent a direct safety risk. The attacks require only a $20 heat lamp and aluminum foil, making them accessible to unsophisticated adversaries. The calibration attack is particularly concerning because it produces delayed effects -- an attacker can set up the heat source, remove it, and trigger fake obstacle detection minutes later when no attack infrastructure is visible.

For thermal camera manufacturers (especially FLIR, which holds 40% market share), the proposed signal processing mitigations should be integrated into firmware. The interdependence of vulnerabilities means partial fixes may worsen other attack vectors.

For sensor fusion architects, these attacks bypass the thermal modality entirely -- a fused system that relies on thermal cameras for nighttime or adverse weather detection may miss obstacles or detect phantom obstacles in exactly the conditions where thermal is most relied upon.

For security researchers, the calibration vulnerability is universal to thermal cameras (all bolometer-based sensors require calibration), suggesting similar attacks may affect thermal cameras across all applications: surveillance, industrial monitoring, building security, and military systems.

Key Takeaways

• Three novel vulnerabilities in thermal camera signal processing enable obstacle misdetection and fake obstacle injection
• Attacks require only a $20 reptile heating lamp and aluminum foil -- cheap, accessible, and invisible (infrared)
• The calibration attack produces delayed artifacts appearing 10 seconds to 2+ minutes after attack termination
• Tested on three commercial cameras (FLIR Boson, InfiRay T2S, FPV XKC130) in real driving scenarios at 40 km/h
• Vulnerabilities are interdependent: fixing one can strengthen others, requiring simultaneous mitigation
• Signal processing mitigations achieve 96-100% effectiveness without degrading genuine obstacle detection
• The calibration vulnerability is universal to all bolometer-based thermal cameras

About the Speaker(s)

Sri Hrushikesh Varma Bhupathiraju is a researcher working on autonomous systems security, sensor perception vulnerabilities, and adversarial attacks on physical-world sensing systems. The research combines signal processing analysis, real-world attack demonstrations, and mitigation design across multiple commercial thermal camera platforms.