Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation

César Sabater (Paper author · INSA Lyon), Sonia Ben Mokhtar (Paper author · INSA Lyon, CNRS), Jan Ramon (Paper author · INRIA Lille)

Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 2 · Moose

Overview

In an increasingly data-driven world, the ability to derive insights from distributed datasets without compromising individual privacy has become paramount. This talk, presented by César Sabater from Inria Lyon, alongside co-authors Sonia Ben Mokhtar and Jan Ramon, addresses a critical challenge in this domain: private mean estimation in fully decentralized systems. The core problem revolves around enabling multiple parties, each holding a private vector (such as a machine learning model or gradients), to collectively compute the average of these vectors without revealing sensitive information about any individual's contribution. This primitive is fundamental to various privacy-preserving distributed computations, including federated learning.

Watch on YouTube · Slides

Visual summary for Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation by César Sabater, Sonia Ben Mokhtar, Jan Ramon
Visual summary for Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation by César Sabater, Sonia Ben Mokhtar, Jan Ramon

Key moments

  1. 0:00 Introduction to private mean estimation and adversary model
  2. 2:00 Limitations of current private mean estimation techniques
  3. 3:30 Foundation: Gossip protocols for decentralized mean estimation
  4. 4:20 Our protocol's innovations: canceling noise, incremental injection
  5. 6:00 Privacy analysis and achieving central DP accuracy
  6. 7:20 Experimental results: improved accuracy and runtime efficiency

Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation

Speakers: César Sabater (Paper author, INSA Lyon); Sonia Ben Mokhtar (Paper author, INSA Lyon, CNRS); Jan Ramon (Paper author, INRIA Lille)

Conference: PETS

YouTube: https://www.youtube.com/watch?v=Dl662jsep0Y

Overview

In an increasingly data-driven world, the ability to derive insights from distributed datasets without compromising individual privacy has become paramount. This talk, presented by César Sabater from Inria Lyon, alongside co-authors Sonia Ben Mokhtar and Jan Ramon, addresses a critical challenge in this domain: private mean estimation in fully decentralized systems. The core problem revolves around enabling multiple parties, each holding a private vector (such as a machine learning model or gradients), to collectively compute the average of these vectors without revealing sensitive information about any individual's contribution. This primitive is fundamental to various privacy-preserving distributed computations, including federated learning.

The research tackles the limitations of existing privacy-preserving techniques, particularly their susceptibility to participant dropouts or crashes in decentralized environments. While Differential Privacy (DP) offers a strong theoretical framework for quantifying information leakage, practical implementations struggle to balance utility, scalability, and robustness. The presented work introduces a novel protocol built upon gossip protocols that achieves a high degree of privacy (close to central DP) and scalability, crucially maintaining its accuracy even when a significant fraction of parties fail or disconnect during the computation.

This work is significant because it bridges a critical gap between the theoretical guarantees of differential privacy and the practical realities of distributed systems. By offering a fully decentralized, dropout-robust mechanism for private mean estimation, it empowers the deployment of privacy-preserving machine learning and analytical tasks in real-world scenarios where a trusted central aggregator is unavailable or undesirable, and where network instability or participant churn is expected. The implications are profound for privacy-preserving AI and collaborative data analysis across various industries.

Background

▶ Watch: Introduction to private mean estimation and adversary model (0:00)

The landscape of private mean estimation is traditionally bifurcated by the assumptions made about the adversary and the system architecture. On one end of the spectrum lies Local Differential Privacy (LDP). In LDP, each party adds noise to its data locally before sharing it, providing strong privacy guarantees against even a powerful adversary who might observe all communications. However, this strong privacy often comes at a significant cost to data utility, leading to highly inaccurate results, especially for complex aggregations like mean estimation.

At the other end is the Central Differential Privacy (CDP) model. Here, a single, trusted central party collects all private data, adds noise to the aggregate, and then releases the noisy result. CDP offers the ideal privacy-utility trade-off, achieving high accuracy with minimal noise. The critical flaw, however, is its reliance on a trusted central party, an assumption that is often unrealistic or undesirable in many real-world applications due to single points of failure, trust concerns, or regulatory constraints.

Recognizing the limitations of these extremes, the research focuses on an intermediate adversary model: a fully decentralized setting where a fraction rho of parties might be corrupted. In this model, honest parties communicate over secure channels, but if a message is sent to or from a corrupted party, the information exchange is compromised. This model is more realistic for many distributed applications, including federated learning, where participants might be untrusted or prone to failure.

Existing solutions within this intermediate model face their own set of challenges. Cryptographic primitives, such as Secure Aggregation, can achieve accuracy very close to central DP. However, these methods typically require a server to orchestrate the aggregation and involve computational work that scales linearly with the number of participating parties. This makes them computationally expensive and not scalable to large numbers of parties, hindering their adoption in massive distributed systems.

Another family of solutions employs purely statistical noise to achieve privacy. These methods generate a correlated structure of noise that effectively hides intermediate values without significantly impacting the utility of the final aggregate. They are generally more accurate and scalable than cryptographic approaches. The Achilles' heel of these statistical methods, however, is their extreme sensitivity to dropouts or crashes. If parties fail in the middle of the computation, the carefully constructed correlated noise distribution breaks, leading to a significant and often catastrophic degradation in accuracy. This vulnerability makes them impractical for real-world distributed systems where participant churn and network instability are common occurrences.

The presented work directly addresses this critical gap, aiming to develop a protocol that combines the desired attributes: high accuracy (close to central DP), scalability to a large number of parties, and crucially, robustness to participant crashes and dropouts in a fully decentralized environment. The foundation for this novel approach is built upon gossip protocols, which inherently support decentralized operations.

Key Findings

▶ Watch: Foundation: Gossip protocols for decentralized mean estimation (3:30)

The central contribution of this research is a novel dropout-robust mechanism for differentially private and fully decentralized mean estimation. This protocol effectively addresses the long-standing trade-off between privacy, utility, scalability, and robustness in distributed systems, particularly in the presence of participant failures. Built upon gossip averaging, a primitive known for its decentralized computation of averages, the protocol introduces three key ingredients to privatize this process while ensuring robustness.

The first crucial ingredient is the addition of canceling noise terms, referred to as "red noise." This noise is strategically added on top of each party's private value. Its primary purpose is to protect intermediate results during the decentralized aggregation process. As values mix across the network through gossip exchanges, this red noise progressively cancels out. The ingenious design ensures that by the end of the computation, the remaining signal is sufficiently mixed and clean, without exposing sensitive intermediate information, yet retaining the desired privacy guarantees.

The second and perhaps most innovative ingredient for dropout robustness is incremental injection. To mitigate the devastating impact of crashes on the variance of correlated noise, the protocol injects each party's private value "bit by bit" into the computation. This means that at any given time, only a small fraction of a party's sensitive information is exposed within the system. Consequently, the amount of noise required to satisfy the differential privacy guarantee, particularly the variance of the correlated noise, is significantly reduced. If a party crashes, the harm to the overall accuracy and privacy is minimized because only a small portion of its data has been injected, limiting the potential for noise distribution breakdown.

Finally, the protocol incorporates some independent noise, termed "blue noise." This is the baseline noise that remains in the ideal scenario where no crashes occur. Its magnitude is carefully calibrated to be on the order of central DP, meaning that in the absence of dropouts, the protocol achieves an accuracy level comparable to the theoretical optimum of central differential privacy. The combination of these three noise mechanisms—canceling noise for intermediate protection, incremental injection for dropout robustness, and independent noise for baseline central DP accuracy—forms the core of their innovative privacy-preserving strategy.

The privacy of this protocol was rigorously studied under an adversary model where the adversary can observe the weights of aggregation, effectively allowing for traffic analysis to infer interactions. The researchers demonstrated that such an adversary can construct a system of linear equations where private values and noise terms are the unknowns. The degree of undeterminacy in this system directly correlates with the level of privacy. A key theoretical finding is that the set of solutions for this system is intimately related to the vector of outgoing nest edges (denoted IIT for party I at iteration T). The protocol's privacy guarantee hinges on the existence of a sufficiently large number of these IIT vectors whose messages have not been seen by the adversary and are linearly independent. Under these conditions, the "blue noise" (independent noise) is bounded, leading to an accuracy very close to central DP when no dropouts occur. Crucially, in the presence of dropouts, the privacy and accuracy degrade gracefully and gradually, a significant improvement over prior statistical methods that experience catastrophic failure. The impact of the correlated noise was further analyzed empirically to validate these theoretical insights.

Empirical comparisons further solidify the protocol's superiority. When benchmarked against other correlated noise techniques (specifically "per-core" protocols), their mechanism consistently exhibited less error across a wide range of settings. This advantage held even in scenarios optimized for the competing protocols, demonstrating the inherent robustness and accuracy benefits of their approach. Furthermore, these comparisons were conducted under fair conditions, ensuring that all techniques used the same communication overhead. The results consistently showed their protocol achieving accuracy very close to central DP.

Finally, the research also compared the runtime performance against popular secure aggregation techniques based on cryptographic primitives. The findings illustrated a significant practical advantage: while cryptographic methods are often described as "constant round," the actual processing required per party scales substantially with the number of participants. Their decentralized, statistical noise-based protocol demonstrated a substantially lower runtime, highlighting its superior scalability and practicality for large-scale deployments where cryptographic overhead becomes prohibitive. This comprehensive set of findings positions the presented protocol as a leading solution for privacy-preserving, dropout-robust mean estimation in fully decentralized environments.

Technical Deep Dive

▶ Watch: Our protocol's innovations: canceling noise, incremental injection (4:20)

The proposed protocol fundamentally builds upon gossip averaging, a powerful primitive for decentralized computation. In a standard gossip averaging protocol, each party i maintains a local estimate x_i of the global average. Initially, x_i is set to its own private value. In each iteration, a party i randomly selects an outgoing set of neighbors and sends its current estimate to them. Simultaneously, it receives estimates from an incoming set of neighbors. Upon receiving these estimates, party i performs a weighted aggregation of its own estimate and the received estimates to compute a new, updated estimate. If these weights are properly calibrated (e.g., using Metropolis-Hastings weights or similar schemes), the local estimates of all honest parties rapidly converge to the true average of all initial private values. While fast and fully decentralized, this basic gossip averaging is inherently not private, as intermediate estimates directly expose sensitive information.

To privatize this process and ensure robustness against dropouts, the authors introduce three synergistic components:

  1. Canceling Noise Terms (Red Noise):

Each party i does not directly inject its private value v_i into the gossip process. Instead, it generates a random noise term r_i (the "red noise") and adds it to its private value. So, the value initially used in the gossip process is v_i + r_i. The crucial design aspect is how these r_i terms are generated and managed. They are designed to be correlated across the network such that when different parties' estimates (which contain these noise terms) are aggregated, the r_i terms progressively cancel each other out. The goal is that the sum of all r_i terms across the network approaches zero. This mechanism serves to protect intermediate values during the mixing process. An adversary observing any single intermediate message would see v_j + r_j, where r_j obfuscates v_j. However, as these noisy values combine and mix, the canceling property ensures that the noise terms vanish from the aggregate, leaving a clean average with privacy preserved by other means. The "sufficiently mixed" condition means that by the time the red noise has largely canceled, the individual contributions have been diffused enough that reverse-engineering original values is computationally infeasible for an adversary.

  1. Incremental Injection:

This is the core innovation for dropout robustness. Instead of injecting the full private value v_i (even with red noise) at the start, parties inject their private value bit by bit into the computation over multiple iterations. For example, in iteration t, a party might only inject a small fraction delta_t v_i into its estimate. This strategy significantly reduces the amount of sensitive information exposed at any given point in time. Consequently, the variance of the correlated noise required to achieve the differential privacy guarantee can be kept much smaller. If a party i crashes at iteration k, only the fraction sum_{t=1 to k} delta_t v_i has been injected. This limited exposure means that the disruption to the overall noise structure and the harm to the final accuracy are greatly minimized compared to scenarios where the entire v_i is injected upfront, and a crash then breaks the noise correlation for the full value. This makes the protocol highly resilient to network churn and participant failures, which are common in real-world decentralized settings.

  1. Independent Noise (Blue Noise):

In addition to the canceling noise terms, a small amount of independent noise (the "blue noise") is added. This noise is typically drawn from a Laplacian or Gaussian distribution, calibrated to the desired privacy budget epsilon and delta. This independent noise acts as a baseline privacy mechanism. In the ideal scenario where no dropouts occur and the canceling noise terms perform perfectly, this blue noise is what ultimately guarantees the differential privacy of the final aggregate. Its magnitude is designed to be on the order of Central Differential Privacy (CDP), meaning that in the absence of failures, the protocol achieves an accuracy level very close to the theoretical optimum for privacy-preserving aggregation.

The privacy analysis of this protocol considers an adversary who can observe the weights of aggregation in the gossip process. This allows the adversary to perform traffic analysis and infer the interactions between parties. Given this information, the adversary can construct a linear system of equations where the private values v_i and the various noise terms (r_i and the independent noise components) are the unknowns. The security and privacy of the system are directly related to the undeterminacy of this linear system; the more solutions that satisfy the equations, the more uncertain the adversary is about any individual v_i.

A key insight from their privacy proof relates to the vector of outgoing nest edges, denoted as IIT (for party I at iteration T). This vector essentially captures the communication pattern of party I at iteration T. The researchers proved that if there is a sufficiently large number of these IIT vectors whose messages have not been observed by the adversary, and these vectors are linearly independent, then the independent "blue noise" term that remains in the system will be bounded. This bounding ensures that the protocol achieves an accuracy very close to that of Central Differential Privacy. Crucially, the protocol is designed such that even if dropouts occur, the degradation of privacy and utility is gradual. This contrasts sharply with prior statistical correlated noise schemes, where a single dropout could catastrophically break the noise correlation, leading to a complete loss of privacy or utility. The empirical analysis further validates how the impact of the correlated noise gracefully degrades with increasing dropouts, confirming the robustness claims.

Demo / Proof of Concept

▶ Watch: Privacy analysis and achieving central DP accuracy (6:00)

While the talk did not describe a live software demonstration or a specific "proof of concept" in the traditional sense of a standalone executable system, the researchers provided extensive empirical validation of their protocol's performance and robustness. These empirical comparisons serve as a robust demonstration of the mechanism's efficacy against existing state-of-the-art techniques.

The primary validation involved accuracy comparisons against other correlated noise techniques, specifically those referred to as "per-core" protocols in the literature. The presented results consistently showed that their novel protocol achieved less error in mean estimation across a wide array of experimental settings. This superior accuracy was evident even in scenarios that were designed to be highly optimistic for the competing "per-core" protocols. A crucial aspect of these comparisons was ensuring fairness: all evaluated techniques utilized the same communication overhead, meaning any observed accuracy improvements were due to the intrinsic design of their protocol rather than increased network traffic. The empirical data strongly supported the claim that their method approaches central DP accuracy very closely, even in the presence of dropouts, a significant improvement over prior statistical methods that are highly sensitive to participant failures.

Furthermore, the researchers illustrated the practical efficiency of their protocol through runtime comparisons against popular secure aggregation techniques that rely on cryptographic primitives. While cryptographic secure aggregation methods are often lauded for being "constant round" (meaning the number of communication rounds does not increase with the number of parties), the actual computational overhead per party can be substantial. The talk highlighted that these cryptographic approaches incur a larger runtime due to the intensive processing required for each party, especially as the number of participants grows. In contrast, their gossip-based, statistical noise protocol, being fully decentralized and relying on lighter statistical operations, demonstrated significantly lower runtime. This makes the proposed mechanism highly scalable and practical for large-scale distributed systems, where the computational burden of cryptographic operations can become a bottleneck. These empirical validations collectively serve as a powerful proof of concept, demonstrating the protocol's superior accuracy, dropout robustness, and efficiency compared to both statistical and cryptographic alternatives.

Defensive Implications

▶ Watch: Experimental results: improved accuracy and runtime efficiency (7:20)

The "Dropout-Robust Mechanisms for Differentially Private and Fully Decentralized Mean Estimation" protocol offers significant defensive implications for organizations and researchers engaged in privacy-preserving distributed computations, particularly in federated learning, collaborative data analytics, and decentralized AI.

  1. Enabling Decentralized Privacy-Preserving Systems: For organizations that cannot or do not wish to rely on a trusted central server (e.g., due to regulatory constraints, trust issues, or single points of failure), this protocol provides a viable and robust alternative. It empowers the deployment of truly fully decentralized privacy-preserving applications, aligning with principles of data sovereignty and distributed trust.
  1. Robustness Against Real-World Failures: A major weakness of prior statistical differential privacy mechanisms was their fragility in the face of participant dropouts or crashes. This new protocol directly addresses this by offering dropout robustness. Defenders can now design systems that can tolerate network instability, node failures, or participants disconnecting without catastrophic loss of privacy or utility. This greatly enhances the reliability and practical applicability of privacy-preserving distributed systems.
  1. Achieving High Accuracy with Decentralization: The protocol demonstrates that it is possible to achieve accuracy levels very close to Central Differential Privacy even in a fully decentralized and dropout-prone environment. This is a critical advantage, as it means organizations do not have to sacrifice significant utility for decentralization and robustness, a common trade-off in previous approaches. This allows for more effective machine learning models and more accurate aggregate statistics.
  1. Scalability and Efficiency: Compared to cryptographic secure aggregation techniques, this statistical noise-based protocol offers superior scalability and lower runtime. For large-scale deployments involving hundreds or thousands of participants, the reduced computational overhead translates into more efficient operations, lower resource consumption, and faster convergence times for tasks like federated learning. Defenders can thus deploy privacy-preserving solutions at scale without encountering prohibitive performance bottlenecks.
  1. Design Principles for Future Systems: The core mechanisms—canceling noise terms, incremental injection, and strategic use of independent noise—offer valuable design principles for future privacy-preserving protocols. The incremental injection strategy, in particular, provides a blueprint for how to build robustness into distributed privacy mechanisms by limiting the exposure of sensitive information at any given time.

Defenders should consider integrating such dropout-robust, decentralized DP mechanisms when designing any system where sensitive data from multiple parties needs to be aggregated privately, especially in environments characterized by untrusted participants or unreliable connectivity. This research provides a crucial tool for building more resilient, private, and efficient distributed systems.

Key Takeaways

  • A novel, fully decentralized, dropout-robust mechanism for differentially private mean estimation was presented, addressing critical challenges in distributed privacy.
  • The protocol achieves accuracy levels very close to Central Differential Privacy (CDP), significantly outperforming Local Differential Privacy (LDP) and existing dropout-sensitive statistical methods.
  • Key innovations include canceling noise terms for intermediate privacy, incremental injection of private values for dropout robustness, and the strategic addition of independent noise for baseline privacy.
  • Empirical evaluations demonstrate that the protocol has less error than other correlated noise techniques, even in optimistic scenarios for competitors, and approaches central DP accuracy.
  • It offers superior scalability and significantly lower runtime compared to cryptographic secure aggregation methods, making it practical for large-scale deployments.
  • This work is crucial for enabling robust and efficient federated learning and other privacy-preserving distributed computations in real-world settings with participant churn and no trusted central party.

About the Speaker(s)

César Sabater is a paper author and presented this work. At the time of the talk, he was a postdoc at Inria Lyon, where this research was conducted. His work focuses on developing robust and private mechanisms for distributed computations.

Sonia Ben Mokhtar is also a paper author and is the head of the LIRIS lab at Inria Lyon, as well as a researcher at CNRS. Her expertise lies in distributed systems, data privacy, and security.

Jan Ramon is a paper author and a researcher at Inria Lille. His contributions to the work likely stem from his background in machine learning, privacy, and algorithms, particularly in the context of distributed and decentralized systems.

Reviews

Dr. Zero (Offensive Security Researcher) — STRONG ACCEPT

This is real, math-heavy privacy engineering that actually acknowledges how distributed systems fail. Sabater and team fixed the fatal flaw in statistical noise for decentralized federated learning: node dropouts nuking the correlation. They achieve Central DP accuracy without the CPU choke of cryptographic Secure Aggregation, which is exactly what builders need right now.

Heather Calloway (CISO) — SOLID

This is academic cryptography research solving a hard problem in federated learning: how to train shared models without pooling data, even when partners drop offline. It isn't ready for enterprise deployment today, but it signals that secure, decentralized data collaboration between untrusted entities is becoming mathematically viable. Keep this on the radar for your long-term AI strategy.

→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026

All talks from Privacy Enhancing Technologies Symposium (PETS) 2026