DorPatch: Distributed and Occlusion-Robust Adversarial Patch to Evade Certifiable Defenses

DorPatch: Distributed and Occlusion-Robust Adversarial Patch to Evade Certifiable Defenses

Speakers: Chaoxiang He

Conference: NDSS Symposium

YouTube: (no public video)

Overview

Deep neural networks (DNNs) have achieved widespread success, yet they remain highly susceptible to adversarial attacks. Among these, adversarial patch attacks are particularly concerning due to their physical realizability – a crafted sticker or paint applied to an object can cause a DNN to misclassify it. This talk introduces DorPatch, a novel and highly effective adversarial patch attack designed to challenge the integrity of state-of-the-art certifiable defenses. Specifically, DorPatch aims to evade defenses like PatchCleanser 64, which provide mathematical guarantees of robustness by assuming patches are spatially bounded.

The core motivation behind DorPatch stems from two critical questions: Can certifiable defenses be circumvented if the spatial boundedness assumption is removed? And, more alarmingly, can adversarially patched examples be certified as robust by these defenses, thereby creating a false sense of security? DorPatch addresses these by generating a fully optimized, distributed, occlusion-robust, and inconspicuous adversarial patch. Its methodology incorporates techniques such as group lasso for sparsity, image dropout for occlusion robustness, density regularization for wide distribution, and a structural loss for perceptual quality. Presented by Chaoxiang He, this research highlights a significant vulnerability in the real-world deployment of DNN models, urging the development of more sophisticated and robust defenses.

Background

[▶ Watch: Introduction to adversarial patches and certifiable defenses (0:00)]()

The landscape of adversarial attacks and defenses has evolved significantly. Early adversarial patch attacks focused on localized perturbations, such as the adversarial eyeglass frame 54 for face recognition or universal targeted patches 6. Subsequent efforts like LaVAN 30 and LOAP 46 optimized patch locations. However, these early patches were often conspicuous. Researchers then moved towards more subtle designs, with RP2 19 generating graffiti-like stickers, TNT Attack 15 using generative models, and IAP 3 employing multi-scale generative models for inconspicuous patches. While most assume a white-box threat model, black-box attacks 68, 58 and extensions to other DNN tasks like object detection 28, 39, 71 and semantic segmentation 50 also exist.

Defenses against these attacks broadly fall into two categories:

• Empirical Defenses: Initial attempts included Local Gradient Smoothing (LGS) 45 and Digital Watermarking (DW) 25, which aimed to suppress noise or detect sensitive regions. Methods like TaintRadar 36 and Februuss 14 focused on detecting and removing localized patches through inpainting. However, these empirical defenses have largely proven ineffective against adaptive white-box attacks 9, often bypassed by incorporating smoothing into patch generation or using Backward Pass Differentiable Approximation (BPDA) 2.
• Certifiable Defenses: Given the limitations of empirical approaches, certifiable robustness became a key area. Techniques like Interval Bound Propagation (IBP) 23, 44 and Randomized Smoothing 10, 33, 35 offered guarantees but often suffered from scalability issues, conservative bounds, or significant overhead. More relevant to DorPatch are certifiable defenses specifically designed for adversarial patches, including Minority Reports (MR) 42 and PatchGuard++ 65. The most critical target for DorPatch is PatchCleanser 64, a state-of-the-art certifiable defense. PatchCleanser operates by assuming an adversarial patch is spatially bounded. It applies masks to the input image in two rounds, and if consistent predictions are achieved across masked versions, it certifies the input as robust. This reliance on spatial boundedness creates a critical vulnerability if an adversary can craft a patch that defies this assumption.

Prior work also includes techniques like Group Lasso 69, used in StrAttack 67 and SAPF 20 to encourage sparsity and structured perturbations. SAPF 20, in particular, factorizes a patch into a binary mask and a pattern, an approach that inspired DorPatch's methodology. The concept of human perceptibility, often using metrics like local variance 41, also informs the design of inconspicuous patches.

DorPatch operates under a white-box threat model, where the adversary has full knowledge of the DNN model. However, it assumes black-box access to potential defenses, meaning the adversary does not know the specific settings of any defense. Crucially, DorPatch removes the restriction that an adversarial patch must be spatially bounded, allowing modifications to any image pixels. Existing patch attacks typically rely on localized patches, which are both suboptimal in effectiveness and vulnerable to defenses like PatchCleanser that exploit their strong locality. PatchCleanser's assumptions—that the model is robust to small occlusions and that the patch can be fully covered by a small mask—are precisely what DorPatch aims to break. Even distributed patches like RP2 19 fail against PatchCleanser because the masking operation corrupts their adversarial properties. This highlights the need for not just distribution, but also robustness to partial occlusions.

To effectively evade PatchCleanser and maximize attack power, DorPatch is designed with four key properties:

1. Distributed: The patch must be widely spread to prevent full occlusion by small masks, achieved through density regularization.
2. Robust to Partial Occlusions: The patch must remain effective even when partially masked, forcing defenses to use large masks that degrade clean accuracy. This property is critical for achieving false certification and is enabled by image dropout.
3. Fully Optimized: The patch's shape, location, and pixel values are optimized for maximum effectiveness, facilitated by group lasso on the mask and a two-stage adversarial patch generation process.
4. Inconspicuous: The patch should blend naturally with the image, achieved through a novel structural loss that promotes smooth and edge-like structural modifications.

Key Findings

[▶ Watch: Introduction to DorPatch and its key techniques (1:35)]()

DorPatch demonstrates several significant contributions and alarming findings:

• Effective Evasion of Certifiable Defenses: DorPatch successfully evades PatchCleanser 64, a state-of-the-art certifiable defense. In digital domain tests, DorPatch consistently achieved extremely low Robust Accuracy (RA), often near 0%, even when PatchCleanser was active. This starkly contrasts with baseline attacks (LaVAN 30, LOAP 46, IAP 3), which PatchCleanser effectively neutralized.
• False Certification of Adversarial Examples: More critically, DorPatch-patched examples not only cause misprediction but can also be certified as robust by PatchCleanser. DorPatch achieved a high Certified Rate of Patched Examples (CRPE), comparable to or even exceeding the certified rate of benign samples reported by PatchCleanser. This creates a dangerous false sense of security, undermining the very purpose of certifiable robustness.
• Superior Perceptual Quality: DorPatch generated patches that are highly inconspicuous, blending naturally with the image. Compared to traditional attacks that produce visible and suspicious patches (LaVAN, LOAP, RP2 19) or even generative approaches that show abnormal lines (IAP 3), DorPatch produced the most imperceptible perturbations. Saliency maps further confirmed that DorPatch's adversarial regions largely fell within low attention areas.
• Successful Physical-World Attacks: DorPatch proved effective in physical-world scenarios, evading PatchCleanser when patches were printed and applied to real objects (e.g., street signs) and captured under various conditions. Its physical patches were significantly less conspicuous than those from IAP 3.
• Robustness Against Empirical Defenses: DorPatch demonstrated robustness against some empirical defenses, notably achieving a low RA of 19.6% against Digital Watermarking (DW) 25, indicating its resilience to partial removal. While less effective against Local Gradient Smoothing (LGS) 45 (RA 35.9%), this was attributed to DorPatch not being specifically designed to target LGS's gradient-based mechanisms.
• Optimized Attack Performance: The proposed two-stage optimization strategy for DorPatch yielded significantly higher CRPEs compared to the lp-Box ADMM 59 solution used in SAPF 20, demonstrating its empirical effectiveness in achieving false certification.
• Criticality of Key Components: An ablation study confirmed that image dropout is unequivocally critical for attacking PatchCleanser, as its removal severely impaired attack effectiveness (RA restored to >86.7%, CRPE dropped to 0%). Density regularization also played an important role in boosting CRPE, especially for larger mask sizes.

Technical Deep Dive

[▶ Watch: Overview of existing adversarial patch attack methods (2:00)]()

DorPatch's methodology is built upon factorizing an adversarial patch into a binary mask M (determining location, shape, size) and a pattern Δ (pixel values), similar to SAPF 20. This factorization leads to a Mixed Integer Programming (MIP) problem, which DorPatch solves using an efficient two-stage optimization method. The overall objective function is formulated as:

$\min_{M, \Delta} L_{adv} + \lambda_1L_{grp} + \lambda_2L_{den} + \lambda_3L_{str} \quad \text{s.t.} \quad ||X_\Delta - X||_p \le \epsilon$

Where $L_{adv}$ is the adversarial loss (e.g., Carlini & Wagner loss), and $\lambda_1, \lambda_2, \lambda_3$ are weighting parameters.

The core innovations lie in the design of the regularization terms:

1. Density Regularization ($L_{den}$): To ensure the patch is widely and uniformly distributed, the image is divided into A evenly sized sampling regions (e.g., 8x8 regions). $L_{den}$ minimizes the standard deviation of the number of selected patch pixels across these regions, promoting an even spread:

$L_{den} = (1/A) \sum_{a=1}^A (A(M_a) - E_{a'}A(M_{a'}))^2$ (Equation 1, simplified)

Here, $A(M_a)$ is the count of patch pixels in region $a$.

1. Image Dropout: This is crucial for making DorPatch robust to partial occlusions and for achieving certifiable robustness against PatchCleanser. During optimization, random square occlusions are applied to the patched image. A set B of possible occlusion squares (varying sizes, positions) is defined. In each iteration, N occluded images ($N=128$ in experiments) are generated by randomly selecting $n_o$ occlusions from B and removing the corresponding regions from the patched image $X_\Delta$. These $N$ occluded images are then optimized together in a single batch. For misclassification by PatchCleanser, $n_o=1$; for certifiable robustness, $n_o=2$. This forces the patch to remain effective even when parts are masked out, directly countering PatchCleanser's detection mechanism.

1. Group Lasso on Mask ($L_{grp}$): To ensure isolated parts of the patch are large enough and have a regular shape, aiding physical realization, DorPatch enforces group-level inclusion or exclusion of pixels. The image is divided into non-overlapping groups of $K \times K$ pixels (e.g., $7 \times 7$ pixels). $L_{grp}$ minimizes the sum of the L2 norms of the indicator matrices for each group:

$L_{grp} = \sum_{l=1}^m ||M_{G_l}||_2$ (Equation 2, simplified)

Minimizing this term encourages many groups to become zero, effectively removing their pixels from the patch.

1. Structural Loss ($L_{str}$): To make the patch visually inconspicuous by promoting continuous and smooth structures, this novel loss is introduced. It combines a conventional total variation loss (for smoothness) with a minimal variance loss (allowing sharp changes like edges). The product of these terms is weighted by the inverse of the local variance $V_i$ of the input image $X$ at pixel $x_i$, encouraging perturbations in complex regions:

$L_{str} = \sum_{x_i \in X_\Delta} \left( \left( \sum_{x_j \in N(x_i)} (x_i - x_j)^2 \right) \min_{x_j \in N(x_i)} (x_i - x_j)^2 \right) / V_i$ (Equation 3)

This ensures that patch pixels are locally smooth, edge points, or in locally complex regions, enhancing inconspicuousness.

The optimization process is divided into two stages:

Stage 1: Generation of Mask M

1. Optimization Using Transparency Mask: The binary mask M is relaxed to a continuous transparency mask M_T with values in $[0, 1]$. The patched image becomes $X_\Delta = (1 - M_T) \odot X + M_T \odot \Delta$. With $M_T$ as a continuous variable, the objective function becomes differentiable, allowing optimization with SGD. $\lambda_2$ and $\lambda_3$ are initially set to $10^{-3}$, while $\lambda_1$ is adaptively adjusted: it starts at 0, then increases (multiplied by 1.2) if adversarialness is maintained, or decreases (divided by 1.3) if lost.
2. Selecting Important Groups to Form Mask M: After optimizing $M_T$, the importance of each group is defined as the sum of $M_T$ magnitudes within that group. Groups are then selected in descending order of importance until the predefined patch budget is met, forming the final binary mask M.

Stage 2: Generation of Pixel Values Δ

Once the binary mask M is fixed, $L_{grp}$ and $L_{den}$ become constant and are removed. The problem simplifies to optimizing $\Delta$ for $L_{adv} + \lambda_3L_{str}$. Here, $\lambda_3$ is dynamically adjusted in a similar fashion to $\lambda_1$ in Stage 1.

For untargeted adversarial patches, DorPatch employs a mixture process. Stage 1 begins with an untargeted substage. Once sufficient adversarialness is achieved, it switches to a targeted substage, selecting the majority label from predictions across different masked positions as the target, ensuring prediction consistency for certification evasion.

For physical-world attacks, additional factors are considered: the patch is constrained within the victim object's bounding box, and Expectation Over Transformation (EOT) 19 is used. EOT calculates $L_{adv}$ as an expectation over a set of random transformations (affine, perspective, color jitter, blurring) applied to $X_\Delta$, making the patch robust to real-world conditions.

Demo / Proof of Concept

[▶ Watch: Understanding certifiable defenses, especially PatchCleanser (3:30)]()

The talk presented compelling evidence of DorPatch's effectiveness in real-world scenarios, effectively acting as a proof of concept for physical-world adversarial patches that evade certifiable defenses.

The physical-world attack performance was evaluated against PatchCleanser using ImageNet models with a 12% patch budget, incorporating EOT 19 to simulate realistic conditions. The experimental process involved:

1. Patch Generation: DorPatch-generated patches were printed onto photographic paper.
2. Physical Application: These printed patches were carefully cut out and then pasted onto physical victim objects, specifically street signs.
3. Data Collection: Photo sequences of the patched signs were captured using an iPhone 13 Pro. These captures were performed under various real-world conditions, including different lighting, angles (up to 30 degrees), and distances (0.4m and 0.7m).
4. Evaluation: The captured frames were then cropped, resized, and fed into the target DNN model for evaluation.

A crucial aspect demonstrated was the perceptual quality of the physical patches. When comparing DorPatch to IAP 3 (a baseline known for inconspicuousness), IAP generated a conspicuous square patch that visibly blurred the word "TIME" on a sign. In stark contrast, DorPatch created distributed patches that resembled green fluorescent stains scattered around the sign's edge. This visual comparison highlighted DorPatch's superior inconspicuousness in a physical setting, making it much harder for human observers to detect.

While the physical deployment of DorPatch's distributed nature was more complex than a single localized patch (taking about 30 minutes for setup compared to 5 minutes for IAP, largely due to precise alignment), the actual sticking of the multiple patch segments took only 1-2 minutes. This demonstrates its practical feasibility despite the increased initial preparation.

The attacking performance in these physical-world tests was robust. Without defense, DorPatch achieved 0% Robust Accuracy (RA) compared to IAP's 16.0%, indicating its superior effectiveness. Even with PatchCleanser defense applied, DorPatch's RA remained very low (2.4% to 1.4% to 2.1%), confirming its ability to evade PatchCleanser in physically realized attacks. This practical demonstration underscores the severe real-world threat posed by DorPatch.

Defensive Implications

[▶ Watch: Defining the white-box threat model for DorPatch (4:00)]()

DorPatch presents a severe challenge to the practical application of DNN models, particularly in safety-critical domains where certified robustness is paramount. Its ability to evade state-of-the-art certifiable defenses like PatchCleanser and, more critically, to achieve false certification of mispredictions, creates a dangerous illusion of security. This directly undermines the fundamental assumption of spatial boundedness that many existing certifiable defenses rely upon.

The work highlights that current certifiable robustness guarantees are insufficient against sophisticated, distributed, and occlusion-robust adversarial patches. New, adaptive defense mechanisms are urgently needed. Several potential countermeasures were discussed:

1. Adversarial Training: Training models with DorPatch-generated adversarial examples could potentially enhance robustness. However, this approach faces significant hurdles. First, the computational cost of generating such complex, optimized patches for training would be extremely high. Second, the talk's evaluation showed that adversarial training with mismatching attacks (e.g., PGD 22 or simple rectangular occlusion) significantly improved robust accuracy for small patch budgets (e.g., 1.5%), but models were still compromised by larger DorPatch examples (e.g., WRN28-4 RA dropped to 13.0% and ResNet110 RA to 0.9% for a 12% patch budget). This suggests that adversarial training is not a complete defense, especially if the training attack does not perfectly match the properties of DorPatch.
2. Specialized Defenses: For very specific scenarios, such as traffic signs containing text, combining Optical Character Recognition (OCR) with the classification model might disable DorPatch. However, such specialized defenses lack generalizability and would only apply to narrow contexts.
3. Protecting Model Access: In the absence of broadly effective general countermeasures, operating DNN models in a black-box setting significantly reduces the risk. The study found that DorPatch's untargeted transfer success rate in a black-box setting was 45.3%. This means that 54.7% of DorPatch's adversarially patched examples lose their adversarialness when attacking a different, unknown model, suggesting that limiting white-box access could serve as a stop-gap measure.
4. Adaptive PatchCleanser: While PatchCleanser can be extended to use multiple masks simultaneously to counter distributed patches, the computational overhead explodes exponentially. Certifying a single sample with two subpatches (Z=2) and default $k=36$ locations requires $1.98 \times 10^5$ inferences for two-round masking. Experiments with single-round masking and dual masks showed only a slight improvement in robust accuracy against DorPatch, which still remained low, indicating this approach is not a scalable or effective solution.

The core implication is that the security community must move beyond the limiting assumption of spatially bounded patches. Future research needs to focus on developing more robust and adaptive certifiable defenses that can inherently account for distributed, occlusion-robust, and inconspicuous adversarial perturbations to truly guarantee the safety and reliability of DNNs in real-world applications.

Key Takeaways

• DorPatch is a novel and powerful adversarial patch attack designed to challenge state-of-the-art certifiable defenses, particularly PatchCleanser.
• It effectively evades certifiable defenses, achieving extremely low robust accuracy even when defenses are active, demonstrating a critical vulnerability.
• DorPatch enables false certification of adversarial examples, making mispredicted results appear robust and creating a dangerous false sense of security.
• The attack is physically realizable, distributed, occlusion-robust, and inconspicuous, making it a practical and stealthy threat in real-world scenarios.
• Key technical components such as density regularization, image dropout, group lasso, and a novel structural loss are essential for DorPatch's efficacy and perceptual quality.
• This work highlights the urgent need for more robust and adaptive defense mechanisms that move beyond the limiting assumption of spatially bounded adversarial patches.

About the Speaker(s)

Chaoxiang He is presented as the speaker for this highly technical and detailed research on adversarial machine learning. Based on the depth and sophistication of the methodology and evaluation, it is clear that Chaoxiang He is a dedicated researcher or academic specializing in the field of AI security and adversarial robustness. The work demonstrates a profound understanding of deep neural networks, adversarial attack techniques, and the intricacies of certifiable defense mechanisms.