SPRINT: Scalable Secure & Differentially Private Inference for Transformers

Francesco Capano (Paper author · SAP SE), Jonas Boehler (Paper author · SAP SE), Benjamin Weggenmann (Paper author · Technische Hochschule Würzburg Schweinfurt)

Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 3 · Moose

Overview

The rapid adoption of large language models and other Transformer architectures has revolutionized many fields, but their deployment often introduces significant privacy and security challenges. This talk, "SPRINT: Scalable Secure & Differentially Private Inference for Transformers," presented by Francesco Capano, Jonas Boehler, and Benjamin Weggenmann, addresses the critical need for protecting both the intellectual property embedded in fine-tuned models and the sensitive data contained within user queries. The research, partially funded by the EU project Gratiation, introduces a novel and practical framework that integrates Differential Privacy (DP) with Multi-Party Computation (MPC) to enable secure and private inference for Transformer models at scale.

Watch on YouTube · Slides

Visual summary for SPRINT: Scalable Secure & Differentially Private Inference for Transformers by Francesco Capano, Jonas Boehler, Benjamin Weggenmann
Visual summary for SPRINT: Scalable Secure & Differentially Private Inference for Transformers by Francesco Capano, Jonas Boehler, Benjamin Weggenmann

Key moments

  1. 0:00 Problem setting and privacy risks in ML as a Service
  2. 2:00 Challenges integrating DP and MPC: noise, errors, overhead
  3. 3:00 SPRINT's core three-step approach: MPC-aware fine-tuning
  4. 5:50 Efficient MPC inference via secret sharing only LoRA matrix B
  5. 7:20 SPRINT's performance: up to 4.2x faster batch inference
  6. 8:20 Accuracy results and gap analysis with MPC approximation
  7. 9:00 Conclusion: Practical, scalable, accurate, open-source solution

SPRINT: Scalable Secure & Differentially Private Inference for Transformers

Speakers: Francesco Capano (Paper author, SAP SE); Jonas Boehler (Paper author, SAP SE); Benjamin Weggenmann (Paper author, Technische Hochschule Würzburg Schweinfurt)

Conference: PETS

YouTube: https://www.youtube.com/watch?v=-VCp4SQXz5g

Overview

The rapid adoption of large language models and other Transformer architectures has revolutionized many fields, but their deployment often introduces significant privacy and security challenges. This talk, "SPRINT: Scalable Secure & Differentially Private Inference for Transformers," presented by Francesco Capano, Jonas Boehler, and Benjamin Weggenmann, addresses the critical need for protecting both the intellectual property embedded in fine-tuned models and the sensitive data contained within user queries. The research, partially funded by the EU project Gratiation, introduces a novel and practical framework that integrates Differential Privacy (DP) with Multi-Party Computation (MPC) to enable secure and private inference for Transformer models at scale.

The core problem arises when model owners fine-tune publicly available pre-trained models on their proprietary, private datasets and then outsource the inference capabilities to a Machine Learning as a Service (MLaaS) provider. This common practice exposes the model weights to the service provider, risking IP leakage, and grants the server access to raw user queries, compromising user privacy. Furthermore, malicious users could exploit this setup to perform membership inference attacks to extract sensitive information about the private training data. SPRINT offers the first practical solution to simultaneously address these multifaceted privacy risks, providing robust guarantees for both the fine-tuning data and the inference process itself.

SPRINT stands out by being a scalable solution that not only offers strong privacy but also maintains high accuracy and delivers significant performance improvements over existing baselines. By carefully balancing the inherent trade-offs between performance, privacy, and accuracy, the framework demonstrates how to adapt Transformer models to be "MPC-aware" during fine-tuning, thereby minimizing the accuracy degradation typically associated with secure computation. Its modular, open-source nature further promotes adoption and continued development in the field of privacy-preserving machine learning.

Background

▶ Watch: Problem setting and privacy risks in ML as a Service (0:00)

The landscape of modern machine learning, particularly with the rise of Transformer models, frequently involves a scenario where a model owner leverages a public, pre-trained model and fine-tunes it using their own sensitive, private data. To monetize or scale this specialized model, the owner often outsources the computational burden of inference to a third-party MLaaS provider. While convenient, this architecture introduces several critical privacy risks. The MLaaS server, having access to the model weights in clear text, poses a risk of intellectual property (IP) leakage. Additionally, the server processes raw user queries, which can reveal sensitive information about the users themselves. A more insidious threat comes from malicious users who might craft specific queries to execute membership inference attacks, aiming to determine if a particular data point was part of the private fine-tuning dataset.

Traditional approaches to mitigate these risks often focus on two distinct, powerful cryptographic and privacy-enhancing techniques:

  1. Differential Privacy (DP): The primary goal of DP is to prevent data leakage from the model's outputs, particularly concerning the data used during its training or fine-tuning. This is achieved by introducing carefully calibrated noise during the fine-tuning process, ensuring that the model's behavior is negligibly affected by the inclusion or exclusion of any single data record. While effective for protecting the training data, DP alone does not protect the model's weights during inference or the user's queries.
  2. Secret Sharing / Multi-Party Computation (MPC): This cryptographic technique ensures the confidentiality of model weights and user queries by distributing data and computations across multiple, non-colluding servers. Data is "secret-shared," meaning no single server holds the complete information. Computations are then performed on these shares, such that the result is computed without any party ever seeing the raw inputs or intermediate values. While MPC provides strong confidentiality, it typically incurs significant computational and communication overhead.

The challenge lies in integrating these two powerful techniques. Combining DP and MPC introduces multiple sources of noise and inefficiency. DP inherently leads to an accuracy drop, and for large Transformer models, this noise scales significantly with model size, requiring more noise for stronger privacy guarantees. MPC, on the other hand, often relies on fixed-point arithmetic protocols for efficiency. This introduces numerical errors due to quantized representation and necessitates approximations for non-linear functions (e.g., ReLU, softmax), which are ubiquitous in neural networks. The most substantial burden, however, is the communication overhead during MPC inference. Large matrix multiplications and interactive non-linear function evaluations require extensive communication between the participating servers, severely impacting performance.

Prior work has either focused on DP for training or MPC for inference, but rarely a practical, scalable integration for Transformer models. This project aims to address this integration challenge, seeking to strike a delicate balance between performance, privacy, and accuracy—a three-way trade-off that has historically limited the practical deployment of fully private and secure machine learning systems. The core idea behind SPRINT is to make the fine-tuning process "MPC-aware," anticipating and mitigating the numerical approximations and efficiency challenges that will arise during the secure inference phase.

Key Findings

▶ Watch: SPRINT's core three-step approach: MPC-aware fine-tuning (3:00)

SPRINT represents a significant leap forward in the field of privacy-preserving machine learning, particularly for Transformer models. The key findings and contributions of this research are multi-faceted:

  • First Practical Solution with Dual Guarantees: SPRINT is presented as the first practical solution that simultaneously provides Differential Privacy (DP) guarantees for the fine-tuning data and Multi-Party Computation (MPC)-based confidentiality for both the model weights and user queries during inference. This integrated approach ensures comprehensive privacy and security throughout the model lifecycle, from training to deployment.
  • Scalability for Transformer Models: Unlike many previous MPC-based solutions that struggled with the size and complexity of large neural networks, SPRINT demonstrates scalability. The evaluation includes results for larger models and datasets, areas where prior MPC work often encountered numerical instability and performance bottlenecks. This makes SPRINT suitable for real-world Transformer applications.
  • High Accuracy with Minimal Gap: A crucial achievement of SPRINT is its ability to maintain high model accuracy even under secure computation. The framework exhibits less than a 1.5% accuracy gap between inference performed in cleartext and that performed using MPC protocols with SPRINT. This gap further reduces to almost zero as the dataset size increases, indicating the robustness of its MPC-aware fine-tuning strategy. While there's an average 2.5% gap compared to non-MPC baselines, this is a significant improvement given the strong privacy guarantees.
  • Significant Performance Gains: SPRINT delivers substantial performance improvements, especially when leveraging modern hardware. For single-sample inference in a LAN environment, it is 1.6 times faster than state-of-the-art baselines. More impressively, by enabling batch inference on GPUs, SPRINT achieves up to 4.2 times faster runtime in a LAN setting, significantly reducing the practical overhead typically associated with MPC. Even in a WAN environment, the speedup is notable, though reduced due to communication latency.
  • Modular and Open-Source Framework: The entire SPRINT framework is designed to be modular and built upon open-source libraries. This open approach encourages further research, allows developers to easily integrate or replace components (e.g., approximation functions, optimizers), and fosters community contributions, accelerating innovation in secure AI.

In essence, SPRINT successfully navigates the complex trade-off between privacy, performance, and accuracy, providing a robust and practical tool for deploying privacy-preserving Transformer models in production environments.

Technical Deep Dive

▶ Watch: Efficient MPC inference via secret sharing only LoRA matrix B (5:50)

The core innovation of SPRINT lies in its three-phase approach, meticulously designed to make the Transformer model "MPC-aware" from the fine-tuning stage, optimizing for both privacy and efficiency during secure inference.

The problem setting involves fine-tuning a public pre-trained Transformer model on private data, then deploying it to an MLaaS provider for confidential inference. SPRINT's solution comprises:

  1. MPC-Aware Pre-processing / Adaptation:

The initial step involves adapting the base model to anticipate the numerical approximations inherent to MPC. Instead of optimizing solely for the new data, the model is trained to be robust to the approximations it will encounter during secure inference.

  • Non-linear Function Approximation: The standard non-linear activation functions (e.g., GELU, ReLU) and the softmax function in Transformers are computationally expensive or difficult to implement efficiently in MPC using fixed-point arithmetic. SPRINT replaces these with floating-point versions of the approximations that will be used during MPC inference. This means the model learns to tolerate these approximations from the outset, minimizing accuracy degradation.
  • Logit Clipping: A technique borrowed from ML literature, Logit Clipping is applied to bound the input values to the softmax function. This results in bounded inputs and more numerically stable results, which is crucial for the fixed-point arithmetic used in MPC.
  • LoRA (Low-Rank Adaptation): SPRINT integrates LoRA, a parameter-efficient fine-tuning technique. LoRA injects small, trainable rank-decomposition matrices into each layer of the pre-trained Transformer model, significantly reducing the number of trainable parameters. This not only speeds up the training process but also has beneficial effects on inference efficiency.
  1. MPC-Aware Fine-tuning with Differential Privacy:

Following the pre-processing, the model undergoes fine-tuning with Differential Privacy guarantees, but with specific modifications that leverage the MPC-aware adaptations.

  • LoRA with Partial Secret Sharing: During fine-tuning, LoRA is applied in a specific manner: the original large weight matrices of the pre-trained model (denoted as matrix A in LoRA literature) are frozen and remain public. Only the newly introduced small LoRA matrix B is trained. Crucially, at the end of fine-tuning, only matrix B contains the private information learned from the sensitive dataset. This is a critical design choice for efficiency during MPC.
  • DP Adam BC Optimizer: To incorporate Differential Privacy, SPRINT uses DP Adam BC. This is an adaptive optimizer that corrects the bias introduced by DP noise in the second moment estimates, leading to improved convergence and better accuracy under DP.
  • Efficient DP Mechanisms: The framework employs Poisson sub-sampling, which is noted as not being as common as shuffling in DP literature but is effectively used here. Additionally, ghost clipping is utilized for efficient gradient clipping, a standard technique in DP training to bound the sensitivity of gradients.
  1. MPC-fication (Secret Sharing for Inference):

The final step prepares the fine-tuned model for secure inference using Multi-Party Computation.

  • Replacing Cleartext Approximations with MPC Protocols: The floating-point approximations of non-linear functions used during fine-tuning are now replaced with their corresponding secure MPC protocols. The paper encourages readers to consult it for specific mathematical formulas, emphasizing the prioritization of numerical stability and accuracy over raw efficiency in these approximations, as the overall architecture provides significant speedups elsewhere.
  • Optimized Secret Sharing Strategy:
  • Embedding Layer Removal: The embedding layers of the Transformer are typically frozen and public. SPRINT capitalizes on this by having Alice (the user's client) compute the embeddings locally. This avoids unnecessary secret sharing and computation for these layers during MPC inference.
  • Partial Secret Sharing of LoRA Matrix B: This is the most significant optimization. As only the small LoRA matrix B holds the private information, only this matrix is secret-shared between the two MPC servers. The much larger base model weights (matrix A) remain in cleartext.
  • Hybrid Multiplication: This partial secret sharing enables hybrid multiplication during inference. The large cleartext base weights (A) can be multiplied with the secretly shared inputs efficiently. Only the multiplication involving the small, secretly shared matrix B requires full MPC protocols. This dramatically reduces the number of expensive MPC multiplications, leading to a substantial speedup.
  • MPC Protocol Details: The MPC inference is implemented as a two-party semi-honest protocol using Crypton. Crypton is a secret sharing framework that leverages fixed-point arithmetic and binary secret shares, providing a robust foundation for secure computation.

By meticulously integrating these techniques, SPRINT effectively balances the performance, privacy, and accuracy trade-offs, making secure and private Transformer inference a practical reality.

Demo / Proof of Concept

▶ Watch: Accuracy results and gap analysis with MPC approximation (8:20)

While the talk did not feature a live, interactive demonstration, the speakers presented a comprehensive evaluation setup and performance validation that serves as a robust proof of concept for SPRINT's capabilities. This evaluation highlights the framework's practical applicability and its superior performance compared to existing solutions.

The evaluation setup utilized a Roberta base model, a widely recognized Transformer architecture, for inference. To assess generality and scalability, experiments were conducted on multiple datasets, including larger ones with substantial validation sets. This is a crucial point, as previous MPC work often struggled with numerical stability and performance when scaling to such large models and datasets.

For Differential Privacy fine-tuning, the experiments were conducted with an epsilon (ε) value of 8, a common privacy budget used in other baselines, allowing for fair comparison. The Multi-Party Computation (MPC) inference was implemented as a two-party semi-honest protocol using the Crypton framework, which employs fixed-point arithmetic and binary secret shares. This setup represents a standard and robust configuration for secure computation.

The results were benchmarked against two distinct types of baselines:

  • MPC baselines: Existing MPC solutions that provide secure inference but lack DP guarantees.
  • DP baselines: Existing DP solutions that protect fine-tuning data but do not offer secure inference via MPC.

SPRINT is unique in providing both guarantees simultaneously.

The performance results demonstrated significant improvements:

  • Inference Speed: When performing single-sample inference in a Local Area Network (LAN) environment, SPRINT was measured to be 1.6 times faster than the state-of-the-art baselines. This speedup was slightly reduced to 1.2 times faster in a Wide Area Network (WAN) due to increased communication latency, but still represents a notable improvement. The most impressive performance gain was observed when enabling batch inference on GPUs. In this scenario, SPRINT achieved a runtime reduction of up to 4.2 times faster in a LAN, showcasing its scalability and efficiency with modern hardware.
  • Accuracy: The talk presented a detailed analysis of accuracy, comparing SPRINT's performance in cleartext versus MPC. A key finding was that there was less than a 1.5% accuracy gap between SPRINT runs performed in cleartext and those performed with MPC protocols. As the dataset size increased, this gap further diminished to almost zero, indicating that the MPC-aware fine-tuning successfully mitigates the numerical challenges of secure computation. When compared to non-MPC baselines, SPRINT exhibited an average accuracy gap of approximately 2.5%. While a gap exists, it is a remarkably small trade-off for the strong privacy and security guarantees provided, especially considering the inherent accuracy drops associated with DP.

These evaluation results strongly validate SPRINT's claim as a practical, scalable, and highly accurate solution for secure and differentially private inference for Transformer models.

Defensive Implications

▶ Watch: Conclusion: Practical, scalable, accurate, open-source solution (9:00)

The SPRINT framework offers profound defensive implications for organizations grappling with privacy and security concerns in the deployment of advanced AI models, particularly those leveraging Transformer architectures. Its integrated approach to Differential Privacy (DP) and Multi-Party Computation (MPC) provides a robust toolkit for bolstering the security posture of AI systems.

For organizations that fine-tune proprietary models on sensitive, private data and then offer them as a Machine Learning as a Service (MLaaS), SPRINT directly addresses critical vulnerabilities:

  • Protection of Intellectual Property (IP): By using MPC, SPRINT ensures that the fine-tuned model weights remain encrypted and distributed across multiple servers, preventing any single MLaaS provider from accessing the model in cleartext. This directly mitigates the risk of IP leakage, safeguarding the significant investment in data and expertise used for fine-tuning.
  • User Query Confidentiality: The MPC protocols extend confidentiality to user queries. Even if an MLaaS provider is compromised, the raw queries remain encrypted, protecting sensitive user inputs from being exposed or misused. This is crucial for applications dealing with personal, financial, or health information.
  • Mitigation of Membership Inference Attacks: The integration of Differential Privacy during the fine-tuning phase provides a provable guarantee against membership inference attacks. This means an adversary, even with full access to the model's outputs, cannot reliably determine whether a specific individual's data was included in the private fine-tuning dataset. This is a powerful defense against a pervasive privacy attack vector.
  • Compliance with Privacy Regulations: By offering provable privacy guarantees (DP) and strong confidentiality (MPC), SPRINT enables organizations to better comply with stringent data protection regulations such as GDPR, CCPA, and HIPAA. It provides a technical mechanism to demonstrate due diligence in protecting user data and intellectual property.
  • Enabling Trust in AI Services: The ability to offer AI services with strong, verifiable privacy and security guarantees can significantly build trust with users and clients. This can be a competitive differentiator, allowing organizations to deploy AI in highly sensitive domains where privacy has traditionally been a barrier.

Furthermore, the modular and open-source nature of SPRINT provides additional defensive advantages:

  • Transparency and Auditability: The open-source codebase allows security researchers, auditors, and privacy experts to inspect the implementation, verify its cryptographic properties, and identify potential weaknesses. This transparency fosters greater trust and allows for community-driven security enhancements.
  • Flexibility for Customization: Organizations can adapt and customize SPRINT to fit specific security requirements, integrate with existing infrastructure, or experiment with alternative MPC protocols or DP mechanisms. This flexibility ensures that the framework can evolve with emerging threats and technological advancements.

In summary, SPRINT provides a robust and practical defensive strategy for securing the entire lifecycle of Transformer models, from the privacy of their training data to the confidentiality of their inference. It transforms a complex multi-faceted problem into a deployable solution that empowers organizations to embrace AI while upholding the highest standards of privacy and security.

Key Takeaways

  • Integrated Privacy and Security: SPRINT is the first practical framework to seamlessly integrate Differential Privacy (DP) for protecting fine-tuning data with Multi-Party Computation (MPC) for securing model weights and user queries during Transformer inference.
  • Scalable Performance: The solution achieves significant performance gains, being up to 4.2 times faster for batch inference on GPUs in a LAN environment compared to existing baselines, making secure AI inference practical for large-scale applications.
  • High Accuracy Maintenance: SPRINT maintains high model accuracy, exhibiting less than a 1.5% accuracy gap between cleartext and MPC inference runs, a crucial factor for real-world usability of privacy-preserving models.
  • MPC-Aware Design: The framework's core innovation lies in its "MPC-aware" fine-tuning, which uses techniques like Logit Clipping and LoRA to prepare models for fixed-point arithmetic and non-linear function approximations encountered during secure computation.
  • Efficient Secret Sharing: SPRINT's efficiency stems from strategically secret-sharing only the small LoRA matrix B (containing private information), while keeping the larger base model weights public. This enables hybrid multiplication and reduces communication overhead.
  • Open-Source and Modular: Built on open-source libraries like Crypton, SPRINT is a modular framework that invites contributions and allows researchers and developers to easily adapt, extend, and audit its components, fostering innovation in secure machine learning.

About the Speaker(s)

The talk "SPRINT: Scalable Secure & Differentially Private Inference for Transformers" was presented by a collaborative team of researchers:

Francesco Capano is listed as a paper author from SAP SE. His work, along with that of his co-authors, was notably funded by the EU under the project Gratiation, highlighting a commitment to advancing privacy-preserving technologies within the European research landscape.

Jonas Boehler is also a paper author affiliated with SAP SE. His contributions are integral to the development and implementation of the SPRINT framework, particularly in addressing the complex challenges of integrating advanced cryptographic techniques with machine learning models.

Benjamin Weggenmann is a paper author from Technische Hochschule Würzburg Schweinfurt. His involvement underscores the collaborative nature of this research, bridging academic rigor with practical industry application, as evidenced by the partnership with SAP SE.

Together, their collective expertise has led to the development of SPRINT, a significant contribution to the field of privacy-preserving machine learning.

Reviews

Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT

Usually when I hear "secure AI," it means some vendor wrapped the OpenAI API in a TLS tunnel and called it a day. SPRINT is actual cryptographically secure engineering. They solved the massive communication bottleneck of Multi-Party Computation by only secret-sharing the LoRA B matrix, making private Transformer inference mathematically viable instead of just an academic thought experiment.

Heather Calloway (CISO) — SOLID

Every board wants to know how we can leverage LLMs on proprietary data without handing our IP to a cloud provider or leaking customer PII. This research presents a scalable, cryptographically secure way to fine-tune and run Transformers without those risks.

→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026

All talks from Privacy Enhancing Technologies Symposium (PETS) 2026