Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference

Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference

Speakers: Zhifan Luo

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=J4yZoU18NXc

Overview

Large language model inference relies on a critical optimization called the key-value (KV) cache, which stores intermediate key and value matrices to avoid redundant computation during autoregressive generation. This talk reveals that the KV cache -- often gigabytes in size and typically processed, transmitted, and stored in plaintext due to performance constraints -- constitutes a dangerous privacy attack surface. The researchers from Georgia Tech demonstrate three distinct attack methods that can reconstruct user input prompts from leaked KV cache data, achieving alarming success rates across multiple model architectures.

The work is particularly relevant to confidential computing scenarios such as NVIDIA Confidential Compute and private cloud inference, where trusted execution environments (TEEs) are used to protect model inference but architectural constraints force the massive KV cache outside the TEE's protection boundary into non-secure memory. The researchers propose KV-Clock, a defense mechanism that provides mathematically lossless protection with less than 1% latency overhead, making it practical for real-world deployment.

Background

▶ Watch: KV cache as a plaintext privacy surface in confidential computing (0:00)

LLM inference is an autoregressive process where each generated token depends on all previous tokens. To avoid recomputing attention for all prior tokens at each step, modern inference systems cache the key and value matrices (the "KV cache") from previous computations. While this dramatically improves throughput, the KV cache is massive -- often gigabytes in size -- and encrypting it would introduce latency that violates the real-time requirements of interactive LLM services.

This creates what the researchers call a "shadow in the system": a plaintext surface that, if accessed by an adversary, could leak sensitive user information. The risk is amplified in emerging confidential model-as-a-service scenarios where cloud providers use TEEs to protect inference computations. In architectures like NVIDIA's confidential computing platform, the KV cache must reside outside the TEE's protection boundary in non-secure memory due to its size, directly exposing it to the cloud service provider.

The threat model assumes a malicious cloud server provider who has access to the KV cache (realistic due to external memory architecture), access to model weights (either because they own the model or can fingerprint an open-source base model), but crucially cannot observe transient runtime operations inside the GPU.

Key Findings

▶ Watch: Chosen-plaintext attack enables near-perfect token recovery (4:00)

The researchers demonstrate three progressively sophisticated attack methods for reconstructing user inputs from KV cache data:

KV Cache Inversion Attack: A direct algebraic approach that exploits the fact that keys and values are linear projections of input embeddings. When projection matrices are square and full-rank, they can be mathematically inverted to recover inputs. However, this attack has strict limitations -- modern models using optimizations like Grouped Query Attention (GQA) create non-square matrices that make direct inversion mathematically impossible, and the attack is primarily effective only on the first layer.

KV Cache Collision Attack: A more universal method that frames the problem as iterative forward-matching rather than reverse engineering. The attacker generates candidate KV caches locally for each possible token and identifies the correct token by finding the mathematically closest match to the leaked target cache. This method works on any layer and any architecture and is optimized using probability-guided pruning and batched outlier detection to reconstruct full inputs in seconds.

KV Cache Injection Attack: A semantic-level attack that appends malicious instructions (such as "repeat the previous content") to the stolen KV cache, tricking the model into leveraging its own capabilities to echo private content stored in the cache.

Across all architectures tested, the collision attack proved universally effective, while the inversion attack was limited to first layers and the injection attack showed high efficiency at extracting semantic meanings.

Technical Deep Dive

▶ Watch: Attack effectiveness heat maps across architectures (6:00)

The collision attack becomes especially powerful when the adversary has chosen-plaintext attack (CPA) capabilities -- the ability to query the model with known prompts and observe the corresponding KV cache. While a naive collision attack can only estimate the distance distribution of incorrect tokens, a CPA-enabled adversary can profile distributions of both correct and incorrect tokens, calculating an optimal decision threshold that achieves near-perfect separation.

The proposed defense, KV-Clock, operates through a three-step process:

1. Linear obfuscation: KV matrices are transformed using secret random invertible matrices S and M, effectively hiding the statistical distribution of the data from the adversary.

1. Blockwise shuffling: A one-time-pad random permutation matrix shuffles the order of row vectors within each block. Critically, the physical memory ordering of the KV cache is redundant for inference because positional semantics are already embedded in the vectors via Rotary Position Embedding (RoPE). As long as internal key-value pair correspondence is maintained, shuffling within blocks does not affect inference output. For a standard block size of 16, this creates a combinatorial barrier of 16 factorial, rendering brute-force matching impossible.

1. Operational fusion: The expensive matrix modification is shifted from the online inference phase to an offline phase by fusing the transformation directly into model weights. For LLaMA 1.8B, this reduces overhead from approximately 7% of precomputation cost to only 0.8%.

KV-Clock achieves what existing defenses cannot: the quality of reconstructed text drops to a level statistically indistinguishable from random noise, while the model's output fidelity is preserved perfectly (mathematically lossless). Unlike differential privacy, which forces a trade-off between utility and privacy, KV-Clock eliminates the signal entirely without degrading model performance.

Demo / Proof of Concept

▶ Watch: Performance results: less than 1% overhead with operational fusion (10:00)

The evaluation was conducted across state-of-the-art model architectures using real-world datasets. Heat maps of reconstruction success showed three clear patterns: the inversion attack succeeds almost exclusively on first layers of legitimate architectures; the collision attack is universally effective (showing red/high-success across all architectures); and the injection attack demonstrates high efficiency at semantic extraction.

When comparing defenses, AES encryption incurs latency nearly equal to the full inference cost itself -- prohibitively expensive. Differential privacy degrades model utility, as the noise required to mask privacy ruins generation quality. HeavyShell, the only prior KV-cache-specific defense, relies on a fixed implementation strategy vulnerable to statistical analysis and is incompatible with RoPE, meaning it cannot work with modern models like LLaMA or DeepSeek. KV-Clock scales linearly with negligible overhead of less than 1%, and in real-world deployments, network latency completely masks the tiny computational cost.

Defensive Implications

▶ Watch: Conclusion: comprehensive offensive and defensive contributions (12:00)

Organizations deploying LLM inference in cloud or confidential computing environments should treat the KV cache as a first-class privacy asset. The assumption that TEE-protected inference automatically protects user data is false when the KV cache resides in non-secure memory.

Existing mitigation approaches are inadequate: encryption (AES) introduces unacceptable latency; fully homomorphic encryption is computationally prohibitive due to the non-linear computations in attention mechanisms; differential privacy destroys model utility; and prior KV-cache-specific defenses like HeavyShell are incompatible with modern architectures using RoPE.

KV-Clock represents a practical path forward, achieving mathematically lossless protection with sub-1% overhead. The key insight -- that positional encoding redundancy in the KV cache allows blockwise shuffling without affecting inference -- is architecturally elegant and compatible with mainstream inference frameworks. Organizations should evaluate KV-Clock integration or equivalent protection for any deployment where KV cache data could be accessed by untrusted parties.

Key Takeaways

• KV cache in LLM inference is typically stored in plaintext and constitutes a critical privacy attack surface, especially in confidential computing deployments
• Three attack methods (algebraic inversion, collision matching, and semantic injection) can reconstruct user prompts from leaked KV cache data
• The collision attack is universally effective across all model architectures and can reconstruct full inputs in seconds
• Chosen-plaintext attack capabilities enable near-perfect token separation through optimal threshold calculation
• Existing defenses (encryption, differential privacy, HeavyShell) all fail to balance security, performance, and model utility
• KV-Clock provides mathematically lossless protection with less than 1% latency overhead by exploiting positional encoding redundancy for blockwise shuffling

About the Speaker(s)

The paper was presented by a surrogate speaker from Georgia Tech on behalf of the authors, who were unable to attend due to unspecified issues. The presenter, Zhifan Luo, delivered a technically detailed presentation covering both the offensive and defensive contributions of the work. The presenter noted they would be unable to answer audience questions, directing inquiries to the original authors.