dp-promise: Differentially Private Diffusion Probabilistic Models for Image Synthesis

dp-promise: Differentially Private Diffusion Probabilistic Models for Image Synthesis

Speakers: Haichen Wang, Nanjing University of Science and Technology; Shuchao Pang, James Cook University; Zhigang Lu, CSIRO's Data61; Yihang Rao; Yongbin Zhou; Minhui Xue

Conference: USENIX Security '24

YouTube: https://www.youtube.com/watch?v=Cz8PutPelwo

Overview

This talk introduces DP-promise, a novel framework for training differentially private diffusion probabilistic models designed for image synthesis. Presented by Haichen Wang from Nanjing University of Science and Technology, alongside collaborators from James Cook University and CSIRO's Data61, the work addresses a critical challenge in the era of data-driven deep learning: the tension between the need for large datasets and the imperative to protect individual privacy. While deep learning models, particularly generative models like Generative Adversarial Networks (GANs) and diffusion models, thrive on vast amounts of data, using sensitive information—such as medical images or facial recognition data—raises significant privacy concerns. Even synthetic images generated by these models have been shown to inadvertently leak information from their original training data, necessitating robust privacy-preserving mechanisms.

DP-promise tackles this problem by proposing an innovative approach that leverages the inherent noise generation within the forward process of diffusion models to achieve differential privacy. Unlike existing methods that predominantly rely on injecting additional noise into gradients via DP-SGD (Differentially Private Stochastic Gradient Descent), which often degrades model performance, DP-promise strategically integrates privacy guarantees directly into the model's architecture. The framework employs a two-phase training process, optimizing the privacy-utility trade-off by carefully managing when and how privacy noise is introduced. This work is crucial for advancing the safe deployment of high-quality generative AI in sensitive domains, ensuring that the benefits of synthetic data can be realized without compromising individual privacy.

The significance of DP-promise lies in its ability to generate high-quality synthetic images while providing strong, theoretically backed privacy guarantees. By demonstrating superior performance across various benchmarks—including sample quality metrics like FID (Fréchet Inception Distance) and Inception Score, and downstream utility measured by classification accuracy—compared to state-of-the-art baselines, DP-promise offers a practical and effective solution for developing privacy-preserving generative models. This innovation is particularly relevant for applications where data sharing and model training are hindered by stringent privacy regulations, opening new avenues for research and development in fields like healthcare, finance, and biometric systems.

Background

▶ Watch: Introduction, privacy challenges in generative models (0:00)

The remarkable advancements in deep learning over the past decade are largely attributable to the availability of massive training datasets. However, this reliance creates a significant hurdle in domains dealing with highly sensitive information, such as medical image processing for diagnostics or face recognition systems for security. In these contexts, assembling large-scale datasets is fraught with privacy concerns, often leading to data scarcity that hampers model development and performance. A common strategy to circumvent direct use of private data is to generate synthetic images that mimic the statistical distribution of the original private dataset. These synthetic images can then be used for training or sharing without exposing real individuals' data.

Despite the promise of synthetic data, recent research has unveiled a critical vulnerability: generative models, including sophisticated architectures like GANs and, more recently, diffusion probabilistic models, are not inherently privacy-preserving. Studies have shown that these models can inadvertently "memorize" specific training examples, leading to membership inference attacks (determining if a specific image was part of the training set) or even reconstruction attacks (recreating original training images from the generated outputs or model parameters). This highlights an urgent need for robust privacy safeguards within the generative model paradigm.

Diffusion models have emerged as a powerful class of generative models, demonstrating superior capability over GANs in producing high-quality and diverse synthetic images. Their operational principle involves two main processes: the forward process and the reverse process. In the forward process, a clean image is gradually perturbed by adding noise over a series of time steps (from 1 to T), transforming it into pure Gaussian noise. Conversely, the reverse process learns to denoise this perturbed image, iteratively removing the introduced noise to recover a clean image. This iterative denoising mechanism is what allows diffusion models to generate highly realistic outputs.

To address the privacy leakage issue, Differential Privacy (DP) has become a gold standard framework. DP offers a strong, mathematically quantifiable guarantee that an individual's presence or absence in a dataset will not significantly alter the outcome of an analysis or model training process. The core idea behind DP is to introduce carefully calibrated noise into the computation, thereby limiting the influence of any single data point. In the context of deep learning, a widely adopted technique is DP-SGD, which adds noise directly to the gradients during the training optimization steps. This ensures that the updates to the model parameters are sufficiently noisy to protect individual privacy.

Prior work on achieving differential privacy in generative models can be broadly categorized into three main approaches: GAN-based methods, feature matching-based methods, and diffusion model-based approaches. While diffusion model-based methods have leveraged DP-SGD to introduce privacy, the authors of DP-promise observe that these existing techniques tend to overlook some inherent privacy features within the diffusion model architecture itself. Specifically, the direct application of DP-SGD can lead to a significant utility degradation, where the quality of the generated images or the performance of downstream tasks suffers due to the added noise. This observation forms the primary motivation for DP-promise, which seeks a more efficient and less intrusive way to integrate differential privacy into diffusion models.

Key Findings

▶ Watch: DP-promise: leveraging inherent noise for privacy (3:30)

The central insight motivating the DP-promise framework is the observation that current differentially private diffusion models, while effective in some respects, often compromise model performance by indiscriminately adding noise to gradients via DP-SGD. The core key finding of this research is that the forward process of diffusion models inherently introduces scaled Gaussian noise to images. This process can be conceptualized as an implicit Gaussian mechanism, which is a fundamental component of differential privacy. By strategically leveraging this inherent noise rather than solely relying on external noise injection, DP-promise significantly improves the privacy-utility trade-off.

The framework's primary contribution is its novel two-phase training approach:

1. Phase I (S to T time steps): In this initial phase, DP-promise harnesses the noise naturally generated during the diffusion model's forward process. The authors demonstrate that by adding scaled Gaussian noise to raw images, differential privacy can be achieved without the need for additional noise from DP-SGD. Crucially, due to the post-processing property of DP, all subsequent training operations within this phase remain differentially private, effectively "saving" the privacy budget that would otherwise be consumed by explicit noise injection. This phase focuses on maintaining model utility by minimizing information loss during private training.
2. Phase II (earlier time steps): For the earlier time steps in the diffusion process, where the inherent noise might not be sufficient to guarantee strong privacy, DP-promise switches to using DP-SGD. This targeted application ensures comprehensive privacy guarantees across the entire training spectrum while optimizing the allocation of the privacy budget.

The research also includes a rigorous theoretical analysis demonstrating how the privacy budget is consumed within the proposed method. This analysis is grounded in Gaussian differential privacy and incorporates subsampling techniques to further enhance privacy guarantees during both training phases. The authors prove that the DP-promise algorithm asymptotically satisfies epsilon-delta differential privacy, providing strong mathematical assurance for the privacy of synthetic images generated by the framework.

Experimentally, DP-promise consistently outperformed all baseline models across a range of crucial metrics. On grayscale datasets like MNIST and Fashion-MNIST, DP-promise achieved lower FID (Fréchet Inception Distance) scores—indicating higher sample quality—and comparable or superior classification accuracy for downstream utility tasks, especially under strong privacy guarantees. For color datasets, it also generated samples closer to original images with low FID and higher Inception Score. A significant finding from the ablation experiments was the confirmation that Phase I, which utilizes the inherent noise, demonstrably enhances both sample quality and downstream utility, validating the core hypothesis of the framework. Overall, DP-promise consistently showed a lower degradation in sample quality when privacy budgets were tightened, showcasing its superior privacy-utility trade-off.

Technical Deep Dive

▶ Watch: Phase 1: Utilizing inherent diffusion noise for DP (4:30)

The technical foundation of DP-promise is built upon a deep understanding of diffusion models and the principles of differential privacy, specifically addressing the limitations of prior work. The talk explicitly outlines a white-box threat model, where an adversary is assumed to have extensive knowledge and access. This includes not only the generated synthetic images but also the complete parameters of the trained generative models. The adversary's objective in this scenario is twofold: either to perform membership inference, determining with high confidence if a particular image was part of the original private training dataset, or to conduct data reconstruction, attempting to reverse-engineer and reproduce the original private images from the model's outputs or parameters. This stringent threat model underscores the necessity for robust privacy guarantees.

The motivation for DP-promise stems from a critical observation regarding existing differentially private diffusion models. These models typically inject noise into the gradients during training using DP-SGD. While DP-SGD is effective in establishing differential privacy, the arbitrary addition of noise can significantly impair the model's ability to learn high-quality representations, leading to a noticeable degradation in the utility of the generated images. The core insight of DP-promise is to exploit a mechanism already present within the diffusion model's architecture.

The forward process of a diffusion model involves progressively adding noise to a data point over a series of time steps. Specifically, at each step t, a small amount of scaled Gaussian noise is added to the image. The authors recognized that this inherent noise addition process can be formally understood as an instantiation of the Gaussian mechanism, a fundamental building block in differential privacy. The Gaussian mechanism achieves DP by adding Gaussian noise to a query's output, with the magnitude of the noise scaled to the query's sensitivity. In the context of the diffusion model's forward process, the "query" can be seen as the transformation of a clean image into a noisy one, and the noise addition inherently provides a degree of privacy.

This realization forms the cornerstone of the DP-promise framework, which is structured into two distinct training phases to optimize privacy and utility:

1. Phase I (from time step S to T): This phase operates during the later stages of the forward diffusion process, where a significant amount of noise has already been accumulated. In this phase, DP-promise applies the standard diffusion model training process without injecting additional DP-SGD noise. Instead, privacy is achieved by leveraging the inherent noise added to the raw images during the forward process. The authors argue that since noisy images are generated by adding scaled Gaussian noise, this process itself confers differential privacy. A crucial aspect here is the post-processing property of DP, which states that any operation performed on differentially private data remains differentially private. Therefore, all subsequent training operations within Phase I, which operate on these inherently noisy and thus private images, maintain the privacy guarantee without needing further explicit noise injection. This strategic approach allows DP-promise to "save the privacy budget" that would otherwise be consumed by DP-SGD, thereby preserving more utility.

1. Phase II (earlier time steps): For the initial time steps of the diffusion process, where images are relatively clean and the inherent noise is less dominant, the privacy guarantees from Phase I alone might be insufficient. To ensure comprehensive privacy across the entire diffusion spectrum, DP-promise switches to using DP-SGD during these earlier time steps. This targeted application of DP-SGD complements Phase I, ensuring that strong differential privacy is maintained even when the images are less perturbed by the inherent diffusion noise. The combination of these two phases creates a more efficient and effective privacy-preserving training mechanism.

The theoretical underpinnings of DP-promise are robust. The authors conducted a detailed theoretical analysis of privacy budget consumption, grounding their work in Gaussian differential privacy. To further enhance privacy guarantees and optimize the privacy-utility trade-off, they employed subsampling techniques during both training phases. Subsampling is a common technique in DP-SGD where gradients are calculated on a randomly selected subset of the data, which inherently reduces the sensitivity of the overall computation and thus allows for a tighter privacy budget or less noise. Through this rigorous analysis, the authors demonstrate that their algorithm asymptotically satisfies epsilon-delta differential privacy, providing a strong mathematical assurance that the synthetic images generated by DP-promise adhere to stringent privacy standards. This theoretical validation is crucial for the framework's credibility and its potential for real-world application in sensitive domains.

Demo / Proof of Concept

▶ Watch: Superior performance on grayscale and Fashion-MNIST datasets (7:40)

While the talk did not feature an interactive, live demonstration in the traditional sense, the presenters effectively provided a proof of concept through extensive experimental results and visualization of synthetic data. These visualizations served as a direct demonstration of the high-quality outputs and privacy-utility balance achieved by the DP-promise framework.

The experiments showcased synthetic images generated by DP-promise alongside those from baseline methods on various datasets. For instance, on grayscale datasets like MNIST and Fashion-MNIST, the visualizations clearly depicted that DP-promise produced high-quality samples that were visually coherent and representative of the original data distribution, even under practical privacy guarantees. This visual evidence supported the quantitative metrics (e.g., lower FID scores) indicating superior sample quality.

Furthermore, for color datasets, the talk presented visualizations that demonstrated DP-promise's ability to generate samples that were "close to original images." These visual comparisons provided a compelling argument for the framework's effectiveness in preserving visual fidelity while ensuring privacy. The ability to generate such high-fidelity images across different data types, especially when contrasted with the often degraded outputs of other differentially private generative models, served as a crucial demonstration of DP-promise's practical utility. The ablation studies, which explored the impact of Phase I, further reinforced this by showing how leveraging inherent noise directly contributed to enhanced sample quality and downstream utility, visually evident in the generated images. These comprehensive visualizations, coupled with quantitative metrics, effectively served as the proof of concept for DP-promise's innovative approach.

Defensive Implications

▶ Watch: Improved privacy-utility trade-off and high-quality samples (8:40)

The DP-promise framework offers significant defensive implications for organizations and researchers working with sensitive data and generative AI. Its primary contribution is providing a robust, theoretically-backed method to generate differentially private synthetic data that maintains high utility, thereby addressing the critical challenge of data privacy leakage from generative models.

Firstly, for data custodians and privacy officers, DP-promise provides a concrete tool to mitigate the risks associated with sharing or training models on sensitive datasets. By employing DP-promise, organizations can generate synthetic versions of their private data that come with strong, quantifiable epsilon-delta differential privacy guarantees. This allows for safer data sharing with external researchers or for internal model development, significantly reducing the risk of membership inference or data reconstruction attacks that plague conventionally trained generative models. This capability is particularly vital in highly regulated sectors such as healthcare, finance, and government, where compliance with privacy regulations (e.g., GDPR, HIPAA) is paramount.

Secondly, for AI/ML engineers and developers building generative models, DP-promise offers a superior alternative to existing DP-SGD-based approaches. The framework's ability to leverage the inherent noise in diffusion models means that privacy can be achieved with less degradation to model performance, including sample quality (FID, Inception Score) and downstream utility (classification accuracy). This translates to more effective and practical privacy-preserving models. Developers can integrate DP-promise into their training pipelines to develop generative AI systems that are both high-performing and privacy-conscious, without having to choose drastically between utility and privacy. This efficiency in privacy budget allocation is a key advantage.

Thirdly, DP-promise encourages the broader adoption of diffusion models in privacy-sensitive applications. Previously, concerns about data leakage might have deterred the use of powerful generative models in domains like medical imaging or biometric data synthesis. With DP-promise, researchers can now confidently explore the capabilities of diffusion models for tasks such as data augmentation, synthetic patient record generation, or privacy-preserving data publication, knowing that individual privacy is mathematically protected.

Finally, the framework's detailed theoretical analysis and experimental validation provide a strong foundation for future research in privacy-preserving machine learning. It highlights the importance of understanding and leveraging inherent properties of model architectures for privacy, rather than solely relying on generic noise injection mechanisms. This could inspire new defensive strategies that are more deeply integrated with specific AI model types, leading to more efficient and less performance-impacting privacy solutions across the board. In essence, DP-promise equips defenders with a powerful mechanism to safeguard individual privacy while harnessing the transformative potential of generative AI.

Key Takeaways

• Innovative Privacy Mechanism: DP-promise introduces a novel approach to differentially private image synthesis by leveraging the inherent scaled Gaussian noise generated during the forward process of diffusion models, reducing reliance on traditional DP-SGD noise injection.
• Optimized Privacy-Utility Trade-off: The framework employs a two-phase training strategy—utilizing inherent diffusion noise in later steps and DP-SGD in earlier steps—to achieve strong privacy guarantees while significantly preserving model utility and sample quality.
• Superior Performance: DP-promise consistently outperforms existing differentially private generative models (GAN-based, feature matching-based, and DP-SGD diffusion models) across various metrics, including lower FID, higher Inception Score, and improved downstream classification accuracy on both grayscale and color datasets.
• Strong Theoretical Guarantees: The method is backed by rigorous theoretical analysis, demonstrating that it asymptotically satisfies epsilon-delta differential privacy based on Gaussian differential privacy and subsampling techniques.
• Enables Safer AI Development: By providing a robust and effective way to generate high-quality, privacy-preserving synthetic data, DP-promise facilitates the secure and ethical deployment of generative AI in sensitive domains like medical imaging and face recognition.
• Reduced Performance Degradation: Unlike many DP-SGD approaches, DP-promise demonstrates a lower degradation in sample quality and utility, even under stringent privacy budgets, making it a more practical solution for real-world applications.

About the Speaker(s)

The primary presenter for this work is Haichen Wang, who is currently affiliated with Nanjing University of Science and Technology. This research, DP-promise: Differentially Private Diffusion Probabilistic Models for Image Synthesis, is a collaborative effort. The joint work involved researchers from Nanjing University of Science and Technology, James Cook University, and CSIRO's Data61, highlighting a multi-institutional collaboration across different regions in the field of privacy-preserving machine learning.