DNN Latency Sequencing: Extracting DNN Architectures from Intel SGX Enclaves with Single-Stepping Attacks

DNN Latency Sequencing: Extracting DNN Architectures from Intel SGX Enclaves with Single-Stepping Attacks

Speakers: Minkyung Park

Conference: NDSS Symposium

YouTube: https://www.youtube.com/watch?v=aS0qX1PJ1V8

Overview

Deep neural network architectures represent valuable intellectual property -- the result of extensive training, research, and computational investment. Confidential AI approaches using trusted execution environments like Intel SGX promise to protect these models from extraction. This talk presents DNN Latency Sequencing (DLS), a new model extraction attack that recovers DNN architecture information (layer types and hyperparameters) from SGX-protected models by analyzing instruction latency traces obtained through single-stepping attacks (SGX-Step).

Presented by Minkyung Park from UT Dallas, the research bridges the semantic gap between raw, noisy latency measurements and high-level architectural information by using execution flows as an intermediate representation. A CNN-BiLSTM model identifies function-level execution flows from latency patterns, and a semi-hidden Markov model recovers basic-block-level execution flows. Evaluated across three DNN libraries (Darknet, TensorFlow Lite, ONNX Runtime) with 100 randomly generated models, the approach achieves over 95% accuracy in identifying layer types and hyperparameters.

Background

▶ Watch: DNN model IP and the confidential AI protection gap (0:00)

DNN model information has three components: structure (number of layers, layer types), hyperparameters (kernel size, pool size, filter counts), and parameters (trained weights and biases). This information is considered valuable intellectual property because training high-quality models requires significant effort and resources. Model extraction attacks attempt to steal this information from deployed models, with two subtypes: architecture stealing (recovering structure and hyperparameters) and parameter stealing (recovering weights and biases).

Architecture information is valuable on its own: it enables training similar models, and serves as a prerequisite for further attacks including parameter stealing and evasion attacks. Confidential AI approaches use trusted execution environments (TEEs) to protect model information by isolating sensitive code and data from untrusted applications and system software. Intel SGX can protect against direct memory access but remains vulnerable to side-channel attacks.

The single-stepping attack (SGX-Step), introduced in 2017, forces an SGX enclave application to exit after every instruction, allowing measurement of instruction latency. A privileged attacker uses a high-resolution APIC timer to measure timing differences between successive enclave exits, producing a sequence of latency traces that encode information about the executed instructions. The challenge is that these latency traces are raw, noisy numeric data, while architectural information is high-level semantic data.

Key Findings

▶ Watch: Key insight: DNN library code structure leaks architecture through execution ... (4:00)

The key insight comes from analyzing real-world DNN library implementations. In Darknet, the main inference function (forward_network) invokes different forward functions for each layer type -- convolution layers call functions f1 through f5, while softmax layers call f6 and f7. If the sequence of executed functions can be identified from latency traces, the model's layer structure can be recovered.

Similarly, in TensorFlow Lite, the convolution function (Conv) contains a for-loop whose iteration count depends on the number of filters, filter height, and filter width. If the sequence of executed basic blocks within each function can be identified, hyperparameters can be inferred from loop iteration counts.

These observations lead to a two-level extraction approach:

1. Function-level execution flows reveal layer types (convolution, pooling, softmax, etc.)
2. Basic-block-level execution flows reveal hyperparameters (kernel size, filter count, etc.)

The approach achieves high accuracy across all three tested libraries:

• More than 95% of function-level execution flows are correctly identified (zero edit distance)
• All layer types achieve more than 96% identification accuracy
• More than 92% of basic-block-level execution flows are correctly identified
• More than 95% accuracy for most hyperparameters, with some exceptions in max pooling layers where patterns are non-unique

Technical Deep Dive

▶ Watch: Semi-hidden Markov model for basic-block-level hyperparameter extraction (8:00)

Stage 1 -- Function-level execution flow recovery (CNN-BiLSTM):

The latency trace segments contain distinguishable visual patterns even though they are noisy. CNN layers are used first to learn these meaningful patterns from the raw latency data. However, patterns alone don't identify functions because multiple functions may share similar operations with identical latencies. Bidirectional LSTM (BiLSTM) layers are then applied to learn the contextual information around each pattern -- the same pattern occurring in different contexts (preceding and following patterns) can be attributed to different functions. The combined CNN-BiLSTM model maps latency segments to function identifiers.

Stage 2 -- Basic-block-level execution flow recovery (semi-hidden Markov model):

Once function-level flows are identified, each function's internal control flow is modeled as a probabilistic state machine. Each basic block becomes a state, control flow transitions become transition probabilities, and the emission probability (probability of observing a specific latency pattern from a given basic block) is calculated using normalized similarity between the basic block's training latency traces and the victim's observed traces. The Viterbi algorithm then finds the most likely sequence of hidden states, recovering the exact basic block execution flow.

Training phase: The attacker trains the CNN-BiLSTM model, semi-hidden Markov models, and mapping tables in advance using training traces collected from known model configurations. This is a one-time effort per target DNN library.

The approach was instantiated on Intel SGX servers with traces collected from Darknet, TensorFlow Lite, and ONNX Runtime. One hundred randomly generated models were used, with 24 layer combinations ensuring valid configurations (e.g., starting with convolution, ending with softmax/cost). Hyperparameters were randomly selected within each configuration.

Demo / Proof of Concept

▶ Watch: Accuracy results: 95%+ on Darknet, TensorFlow Lite, ONNX Runtime (10:00)

The end-to-end accuracy evaluation measured how many layer types and hyperparameters could be correctly recovered per model. Histograms showed that most models achieved more than 95% accuracy across all three DNN library implementations. The evaluation used several metrics to capture different aspects of recovery quality.

Some hyperparameters of max pooling layers showed lower accuracy due to non-unique and common patterns in the latency traces, where different hyperparameter configurations produced indistinguishable timing signatures. This limitation is acknowledged but does not significantly impact overall model recovery.

The research includes artifact evaluation with code and data available via QR code, enabling reproducibility. The attack framework is designed to be extensible to other architectures: the single-stepping attack methodology has been demonstrated on AMD SEV, ARM TrustZone, and Intel TDX, suggesting that DLS could be adapted to these platforms if consistent and deterministic latency traces can be obtained.

Defensive Implications

▶ Watch: Q&A: GPU TEE applicability and obfuscation countermeasures (16:00)

The primary defensive implications center on the limitations of TEE-based confidential AI:

1. Intel SGX alone does not protect model architecture IP. Organizations deploying models in SGX enclaves should not assume the architecture is protected from side-channel extraction.

1. Obfuscation is a potential countermeasure but was not evaluated in this work. Adding no-ops or extra operations to each layer could potentially disrupt the latency patterns, but if the execution flow remains uniquely identifiable after obfuscation, the attack still applies. Effective obfuscation would need to make different layer types and hyperparameter configurations produce indistinguishable latency signatures.

1. Constant-time implementations for DNN inference could mitigate the timing side channel but would likely impose significant performance overhead and may not be practical for real-time inference.

1. Cross-platform awareness: The underlying single-stepping methodology extends beyond SGX to AMD SEV, ARM TrustZone, and Intel TDX. Organizations using any TEE-based confidential computing for model protection should assess their exposure to timing side channels.

Key Takeaways

• DNN architecture (layer types and hyperparameters) can be extracted from Intel SGX-protected models using instruction latency traces from single-stepping attacks
• Execution flows serve as an effective intermediate representation bridging raw latency data and semantic architectural information
• CNN-BiLSTM identifies function-level flows (layer types) with over 96% accuracy; semi-hidden Markov models recover basic-block-level flows (hyperparameters) with over 95% accuracy
• The approach was validated across three DNN libraries (Darknet, TensorFlow Lite, ONNX Runtime) with 100 randomly generated models
• The single-stepping attack methodology extends to AMD SEV, ARM TrustZone, and Intel TDX, suggesting broader applicability
• Code and data are available for artifact evaluation and reproducibility

About the Speaker(s)

Minkyung Park is a researcher at UT Dallas, conducting collaborative research with Purdue University. Park presented the work in person with clear technical explanations, engaging with audience questions about applicability to GPU-based TEEs and obfuscation countermeasures. Park's responses indicated confidence that the execution-flow-based approach could extend to other architectures given consistent latency traces, while acknowledging that obfuscation effectiveness requires further study.