On the Security Risks of Memory Adaptation and Augmentation in Data-plane DoS Mitigation

On the Security Risks of Memory Adaptation and Augmentation in Data-plane DoS Mitigation

Speakers: Hocheol Nam

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=-_U6EYLG1Xo

Overview

Programmable switches have transformed DoS defense by enabling real-time, line-rate detection and mitigation directly in the network data plane. But this talk reveals that the very optimizations making these defenses efficient -- memory slicing, resource augmentation, adaptive memory reallocation, and periodic sketch refresh -- create new exploitable attack surfaces. The researchers introduce the Heracles attack, the first attack that proactively exploits the internal optimization mechanisms of data-plane DoS defenses rather than merely trying to evade detection. Tested against Poseidon (presented at Oakland 2024), Heracles caused the defense to fail to block 78% of malicious traffic using synchronized augmentation and 50% using memory squeezing. The team also proposes SHIELD, a hierarchical sketch-based defense that decouples timing across layers, blocks all malicious traffic within 3 seconds, and is fully open-source on GitHub.

Background

▶ Watch: How programmable switches rebalanced the DoS attacker-defender asymmetry (0:00)

Historically, DoS attackers held an asymmetric advantage: they could generate massive traffic volumes with adaptive attack vectors and precise timing, while defenders were limited by the inherent latency between control and data planes. Programmable switches (like Intel Tofino) rebalanced this asymmetry by allowing customized packet processing at line rate with terabit-scale throughput.

Poseidon, presented at IEEE S&P 2024, demonstrated that data-plane programmability could make DoS defense comparable to the attack in flexibility and speed. It uses catch-based monitoring to track flows with limited on-chip memory, register memory slicing for resource sharing across multiple detection tasks, resource augmentation to offload overflow information to the control plane, adaptive memory slicing to dynamically reallocate counter bits across tasks, and time window-based periodic refresh to maintain sketch accuracy.

Each optimization is individually well-motivated. But when stacked together, they unintentionally create new attack surfaces. The strict hardware constraints of programmable switches -- limited SRAM/TCAM, fixed pipeline stages, single memory access rules -- force these optimizations, making the vulnerability structural rather than accidental.

Key Findings

▶ Watch: Heracles attack: proactively exploiting defense optimizations (4:00)

The Heracles attack uses three strategies to exploit optimization mechanisms in data-plane DoS defenses:

Synchronized Augmentation: The attacker aligns multiple attack flows so their counters overflow simultaneously across all in-network detection tasks. When every task's counter reaches maximum at the same time, it triggers a flood of overflow messages to the control plane. The control plane can only handle a fraction of these messages -- many are dropped or delayed, causing severe undercounting of attack traffic. Against Poseidon, this caused the defense to fail to block 78% of malicious traffic.

Memory Squeezing: The attacker first inflates memory demand for non-target tasks (e.g., tasks A and B), causing the adaptive memory slicer to reallocate counter space away from the target task (D). Once task D's counter space is squeezed, the attacker switches all traffic to task D. The squeezed counters overflow rapidly, generating 800K overflow messages per second to the control plane. The defense fails, allowing approximately 50% of DoS traffic to bypass and reach the target server.

Time Window Exploitation: Attackers probe the defense system to infer the timing of periodic sketch refresh windows. By aligning attack traffic with refresh boundaries, they can exploit the transient accuracy loss during counter resets. This strategy applies broadly to all systems using predefined time windows.

Parameter Inference: The Heracles attack begins with a preparation phase where the adversary probes the system and infers internal parameters from feedback. By varying traffic volume and observing when mitigation triggers, the attacker can accurately infer detection thresholds, time window lengths, and refresh timing -- using only a loosely synchronized botnet (1-2 second synchronization is sufficient).

Technical Deep Dive

▶ Watch: Memory squeezing: forcing counter overflow at 800K messages/second (8:00)

The attack exploits a fundamental tension in programmable switch design: hardware constraints force optimizations that create timing and resource-sharing side channels.

Synchronized Augmentation Attack Flow:

1. Attacker sends multiple attack vectors simultaneously at controlled rates
2. Counters for multiple detection tasks increase in lockstep
3. When all counters reach maximum, simultaneous overflow messages flood the control-to-data plane channel
4. The control plane becomes the bottleneck -- it cannot process all overflow messages
5. Dropped messages cause the defense to lose count of attack packets
6. The defense's mitigation threshold is never reached for the uncounted traffic

Memory Squeezing Attack Flow:

1. Attacker sends traffic counted by non-target tasks to inflate their memory demand
2. Adaptive memory slicing reallocates counter bits to the inflated tasks
3. Target task's counter space shrinks
4. Attacker pivots all traffic to the target task
5. Squeezed counters overflow immediately, generating up to 800K overflow messages/second
6. Control plane drops most messages; defense undercounts and fails to mitigate

Broader Applicability: The same principles extend beyond Poseidon to other in-network DoS defenses: Jaqen and Poseidon are vulnerable to time window exploitation, Ripple and Mu (link flooding attack defenses) are vulnerable to time windowing, and Mu is additionally vulnerable to memory squeezing due to its adaptive memory slicing design.

SHIELD Defense: Uses hierarchical sketch design with multiple layers operating at different granularities:

• Layer 1 uses reset-based refresh with period T1
• Higher layers use decay operations (e.g., counter to 25% every T2, counter to 50% every Tn)
• Decoupled timing across layers obfuscates observable timing side channels
• Mice flows (majority) are handled in lower layers with small counters
• Elephant flows naturally migrate to higher layers with larger counters
• Zero or minimal data-to-control plane communication prevents control plane flooding
• Fixed-size memory slicing prevents memory squeezing

Demo / Proof of Concept

▶ Watch: SHIELD: hierarchical sketch with decoupled timing across layers (10:00)

All attacks and defenses were implemented and evaluated on a physical Tofino switch test bed:

• Synchronized augmentation against Poseidon: defense failed to block 78% of malicious traffic
• Memory squeezing against Poseidon: overflow traffic reached 800K messages/second, ~50% of attack traffic bypassed the defense
• SHIELD evaluation: blocked all malicious traffic within 3 seconds with approximately 2 percentage points lower false positive rate compared to Poseidon, while maintaining strong detection accuracy despite the hierarchical design
• The adversary model uses a realistic loosely synchronized botnet (1-2 second synchronization)

SHIELD is fully open-source on GitHub with functional PTF tests for reproducibility.

Defensive Implications

▶ Watch: Key takeaway: performance-first design creates insecure defaults (14:00)

This research has critical implications for organizations deploying programmable switch-based security:

Performance-First Design Creates Insecurity: The paper's central lesson is that efficiency optimizations under hardware constraints can create structural vulnerabilities. Adaptive memory allocation, resource augmentation to control planes, and predictable refresh timing are all individually reasonable but collectively exploitable. Defense designers must consider adversarial interactions between optimization mechanisms.

Control Plane as Structural Bottleneck: When data-plane defenses rely on control-plane augmentation for overflow handling, the control plane becomes a denial-of-service target itself. SHIELD's approach of eliminating or minimizing control-plane communication for normal operation is the right architectural direction.

Timing Side Channels in Network Defenses: Predictable time windows for sketch refresh create side channels that attackers can probe and exploit. SHIELD's hierarchical design with decoupled timing per layer is a practical mitigation.

Broader P4/Tofino Security: As organizations deploy more security functions on programmable switches (not just DoS mitigation), the same categories of optimization-driven vulnerabilities may emerge in other applications. The security research community should proactively audit in-network security applications for similar structural weaknesses.

Key Takeaways

• The Heracles attack is the first to proactively exploit internal optimization mechanisms of data-plane DoS defenses, not just evade detection
• Synchronized augmentation causes Poseidon to fail to block 78% of malicious traffic by flooding the control plane with overflow messages
• Memory squeezing forces adaptive memory reallocation then pivots to the squeezed task, causing 50% attack traffic bypass
• Attackers can infer internal defense parameters (thresholds, timing, window lengths) through simple probing with only loosely synchronized botnets
• SHIELD's hierarchical sketch with decoupled timing per layer eliminates the timing side channels and control plane bottleneck that Heracles exploits
• The vulnerability is structural: hardware constraints force optimizations that create exploitable side channels across multiple in-network defense systems

About the Speaker(s)

Hocheol Nam (presenting as Hal Nam) is a PhD student at KAIST, working with collaborators Dejan from KAIST, Han Goofy from Texas A&M, and advisor Muk from KAIST. The research focuses on the security of programmable data plane applications, with both offensive (Heracles) and defensive (SHIELD) contributions. The SHIELD implementation is fully open-source for community validation and extension.