DP-fy your DATA: How to (and why) synthesize Differentially Private Synthetic Data: Methods for DP synthetic TABULAR data

DP-fy your DATA: How to (and why) synthesize Differentially Private Synthetic Data: Methods for DP synthetic TABULAR data

Speakers: Natalia Ponomareva, Sergei Vassilvitskii, Peter Kairouz, Alex Bie

Conference: ICML 2025

YouTube: https://slideslive.com/39043838

Overview

This talk, delivered by Natalia Ponomareva, delves into the intricate world of generating Differentially Private (DP) synthetic data, specifically focusing on image and tabular modalities. Building upon previous discussions on the necessity of DP synthetic data and its application to text, Ponomareva provides a comprehensive overview of current methods, their challenges, and their practical implications. The presentation highlights the significant progress made in synthetic data generation, from early Generative Adversarial Networks (GANs) to sophisticated Diffusion Models for images, and from traditional workload-based approaches to emerging Large Language Model (LLM) techniques for tabular data, all while striving to meet rigorous privacy guarantees.

The core motivation behind this research is the ubiquitous need to leverage vast datasets for machine learning advancements without compromising individual privacy. As data generation becomes increasingly pervasive, particularly with user-generated content like images and sensitive personal records in tabular formats, the imperative for robust privacy-preserving techniques grows. Ponomareva's talk is crucial for researchers, practitioners, and data stewards navigating the complex trade-offs between data utility, model performance, computational cost, and privacy assurances in real-world AI/ML applications.

The article explores the technical nuances of various DP synthesis methods, contrasting their strengths and weaknesses across different data types. It underscores that while significant strides have been made, particularly in image generation, the integration of Differential Privacy introduces unique complexities that often lead to a compromise in data fidelity or require substantial computational resources. For tabular data, the evolution from statistical marginal matching to leveraging the semantic understanding of LLMs presents a promising, albeit still developing, frontier.

Background

▶ Watch: Introduction to DP synthetic images and challenges (0:00)

The proliferation of data across nearly every domain has fueled rapid advancements in machine learning. However, this progress often comes at the cost of individual privacy, as raw datasets frequently contain sensitive information. Differential Privacy (DP) has emerged as a gold standard for quantifying and limiting privacy risks, providing a mathematical guarantee that the presence or absence of any single individual's data in a dataset does not significantly alter the outcome of an analysis or model training. This principle is crucial for building trust and enabling responsible data sharing and analysis.

The generation of synthetic data, data that mimics the statistical properties of real data without containing any original records, is a powerful technique for enabling privacy-preserving machine learning. While non-DP synthetic data generation has seen remarkable progress, particularly with models like Generative Adversarial Networks (GANs) and Diffusion Models producing highly realistic images and text, integrating DP into these complex architectures presents unique challenges. The primary difficulty lies in the inherent trade-off: achieving strong privacy guarantees typically requires injecting noise, which can degrade the utility and fidelity of the synthetic data, especially for high-dimensional and complex modalities like images.

For image data, the challenge is compounded by its continuous nature and high dimensionality. Unlike text, where dependencies are largely sequential, pixels in an image depend on their multi-directional vicinity, necessitating very complex models. DP-fying these models, which often have millions or billions of parameters, is difficult because the noise added for privacy can severely distort the intricate patterns required for realistic image synthesis. Early attempts at DP image synthesis often yielded low-resolution or "whimsical" results, highlighting the gap between non-private and private generation quality.

Tabular data, though seemingly simpler, presents its own set of challenges. It often contains a mix of categorical and numerical features, and privacy concerns frequently revolve around preserving marginal distributions or specific relationships between columns, rather than pixel-level fidelity. Traditional approaches have focused on statistical methods, often building histograms and adding noise to aggregate counts. However, these methods can struggle with high-dimensional tables or complex, implicit relationships within the data, leading to a demand for more sophisticated, end-to-end solutions that leverage modern deep learning techniques. The problem in the ML/systems space, therefore, is to develop methods that can generate high-utility synthetic data across diverse modalities while adhering to strict DP guarantees, efficiently and at scale.

Key Findings

▶ Watch: Overview of DP image synthesis techniques (2:30)

Natalia Ponomareva's talk elucidates several key findings and trends in the landscape of DP synthetic data generation for images and tabular data:

1. Image Synthesis Remains Highly Challenging: Despite advancements in non-DP image generation, achieving high-fidelity, high-resolution DP synthetic images is still an active research area. Models like DP-GANs and early DP Diffusion models often produce "whimsical" or low-quality outputs, underscoring the severe impact of privacy noise on visual complexity.
2. Public Pre-training is Crucial: For both image (DP Diffusion) and text (as alluded to by Alex Bie), leveraging publicly pre-trained models (e.g., encoder-decoders, foundational models) significantly improves the quality of DP synthetic data when fine-tuning with private data. This approach helps reduce the amount of noise needed during DP-SGD.
3. Private Evolution (PE) as a Promising Alternative: For image data, Private Evolution stands out as a computationally cheaper method that can work with smaller private datasets (hundreds to a thousand images) and achieve very low epsilon values (e.g., $\epsilon=1$ or $5$). It leverages powerful, publicly pre-trained foundational models for generation, with privacy applied only to a voting mechanism, but relies on the private data distribution being somewhat similar to the foundational model's training data.
4. Workload-Based Methods Excel for Tabular Marginals: For tabular data, traditional workload-based methods (also known as marginal-based methods) are highly optimized for preserving specific marginal distributions and relationships. They are mature, well-understood, and often outperform newer deep learning approaches when the objective is to match these statistical properties.
5. LLMs Show Potential for Tabular Data, but with Nuances: Large Language Models offer a promising avenue for end-to-end tabular data generation by leveraging their pre-training knowledge of column names and values. However, direct application of DP-SGD to LLMs trained on tabular data (like GReaT) doesn't work well. Two-stage training, involving initial format compliance learning on public/random data followed by DP fine-tuning on private data (often with LoRA and specialized losses), is necessary.
6. Importance of Privacy Unit and Metrics: Defining the privacy unit (e.g., user-level vs. example-level) is critical, especially for images. For evaluation, standard metrics like Inception Score, FID, and MAUVE are used for images, while distributional metrics focused on marginal matching are key for tabular data. The best metric remains downstream task performance.
7. Computational Cost and Budget Allocation: DP training, especially with complex models, is computationally expensive. Techniques like LoRA for parameter-efficient fine-tuning and intelligent budget allocation strategies (as seen in workload-based tabular methods) are vital for practical deployment.

Technical Deep Dive

▶ Watch: DP-GANs: architecture, challenges, and limitations (4:00)

The talk provides a detailed exploration of the technical methodologies employed for generating DP synthetic data across two distinct modalities: images and tabular data.

DP Synthetic Image Generation

The speaker first addresses image data, emphasizing its complexity due to continuous pixel dependencies and high dimensionality. A critical initial consideration is the privacy unit. While academic work often treats a single image as the privacy unit, the talk advocates for a user-level privacy unit, where a "user" could be the creator or an actor in the image. This requires defining user sets, performing user contribution bounding, and then treating each image as an example-level unit within its user's contribution.

Techniques for DP image synthesis broadly mimic those for text, falling into DP training/fine-tuning and inference-only methods. Data release algorithms, which compute and DP-fy statistics, have not proven as successful.

1. DP Training and Fine-tuning:

• DP-GANs: Early attempts at DP image generation leveraged GANs, which consist of a generator and a discriminator. The discriminator is the only component that interacts with private data and thus needs to be DP-fied, typically via DP-SGD. Training GANs is inherently difficult due to their adversarial nature and tendency to diverge. DP-fying them exacerbates these issues. Improvements have included:
• Using public data for pre-training feature extractors or encoder-decoders.
• Simplifying adversarial training with alternative losses, such as Wasserstein-1 loss, which generates bounded gradients, easing hyperparameter search.

Despite these efforts, DP-GANs still fall short in quality compared to non-DP counterparts.

• DP Diffusion Models: These models have largely superseded GANs for non-DP image generation and are now central to DP efforts. Diffusion models involve a two-step process: a forward diffusion process (gradually adding Gaussian noise to an image until it becomes pure noise) and a backward sampling process (a neural network learns to reverse the noise addition). Applying DP-SGD to the noise prediction network is a natural approach.
• DP-LDM (Latent Diffusion Models): This method improves upon direct DP-SGD by reducing dimensionality. It uses a publicly pre-trained encoder-decoder to map images to a lower-dimensional latent space. DP-SGD, often with LoRA (Low-Rank Adaptation), is then applied to fine-tune the diffusion model in this latent space, reducing the number of parameters requiring DP noise injection.
• DP-PROMISE: This innovative work leverages the inherent noise addition in the forward diffusion process. It proposes a two-stage fine-tuning. In stages where images already have significant Gaussian noise, the model can derive DP guarantees without explicit DP-SGD, by bounding sensitivity based on image dimensions. DP-SGD is reserved for stages with less noise. This selective application of DP-SGD aims to improve performance by reducing unnecessary noise injection.

Despite these advancements, examples from datasets like "celebrities" still show synthetic images that are "whimsical," indicating ongoing quality challenges.

1. Inference-Only Methods (PATE-style & Private Evolution):

• PATE-style Algorithms: These methods introduce DP at the inference stage. PATE-GAN was an early attempt for images, where a private dataset is split to train multiple "teacher" discriminators without DP. Their aggregate, noisy predictions are then used to train a "student" discriminator. G-PATE extended this by DP-fying gradients in a PATE-like fashion. However, PATE-style methods for images have shown very limited success, with generated MNIST digits still appearing rudimentary.
• Private Evolution (PE): This method is computationally cheaper as it avoids DP training. It relies on a foundational model that provides "random" and "variation" APIs (generating new images or altering existing ones). The process involves:

1. Embedding private data using a feature extractor (e.g., CLIP or Inception Network).
2. Generating synthetic images (randomly or from public data) and embedding them.
3. Each private example "votes" for its closest synthetic counterpart.
4. The histogram of votes is DP-fied.
5. Synthetic images with the highest DP-fied votes are selected, altered using the foundational model's variation API, and the process repeats.

PE's advantages include high-resolution output (from the foundational model), ability to work with small datasets (hundreds of images), and very low epsilon guarantees (e.g., 1 or 5) because only the votes are privatized. The main downside is that the private data's distribution must be similar to the foundational model's training data.

DP Synthetic Tabular Data Generation

Tabular data is represented as a histogram of feature combinations. The goal is to generate a DP version of this histogram or sample from it.

1. Workload-Based (Marginal-Based) Methods: These are the oldest and most well-studied approaches, often explored in the context of query release. They follow a select-measure-estimate paradigm:

• Select: The algorithm determines which marginal queries (predefined relationships or distributions) are most important to measure, doing so with DP guarantees on the error.
• Measure: The selected marginals are measured on the private data, and DP noise is added (e.g., using P-norm, infinity norm, or 1-norm/2-norm).
• Estimate: A probabilistic model or a reconstructed histogram is built from the noisy marginal measurements to explain both measured and unmeasured marginals. This step is computationally intensive, often involving probabilistic graphical models (PGMs) like in Private-PGM or reconstructing residuals as in GREM.

Key axes differentiating these algorithms include:

• Workload awareness: How well the algorithm prioritizes marginals relevant to a user-defined workload (e.g., AIM, REP).
• Data awareness: How the strategy adapts to the statistical properties of the data.
• Budget awareness: How the DP budget is intelligently allocated between select and measure steps (e.g., AIM adapting budget per round, PrivBayes, PrivSyn allocating based on marginal complexity).
• Computational awareness: Ensuring the resulting probabilistic model doesn't become too complex (e.g., MST, PrivMRF, AIM adding marginals conditionally).

1. LLM-Based Methods: The motivation here is to leverage LLMs' pre-training knowledge, which can infer relationships from column names and values (e.g., age and marriage status), unlike histogram-based methods that only discover these given enough data.

• GReaT: An early attempt to generate tabular data end-to-end using LLMs. It encodes each row into a text format (e.g., "education is bachelor's, sex is male, income is less than 50k"), fine-tunes an LLM on this text, and then samples from it. Direct DP-SGD on this approach has not been successful.
• Two-Stage DP-GReaT (Concurrent Works): To address the DP challenge, recent works propose a two-stage training:

1. Format Compliance Learning: The model is fine-tuned on random or public data using real column names and domains. This teaches the LLM the expected format and value types for each column (e.g., "gender" column expects "male" or "female").
2. Private Data DP-SGD: The LLM is then fine-tuned on the private data using DP-SGD, often with LoRA to reduce the number of parameters requiring DP. Key innovations include:

• Weighted cross-entropy loss: Assigns more weight to tokens representing private values (e.g., "35" in "age is 35") and less to formatting tokens (e.g., "age is", commas).
• Numerical understanding loss: Penalizes the magnitude of numerical errors, addressing the issue where LLMs treat "99" and "100" as distinct tokens with similar loss to "1" and "100", despite vastly different numerical distances.

One concurrent work by Fonyo showed that format compliance could even be learned from public data without sharing column names/domains. While these LLM-based methods have shown success against some workload-based methods (e.g., REP), they were outperformed by others like AIM, especially when fidelity is measured by marginal matching, which workload-based methods are optimized for.

Experimental Setup & Results

▶ Watch: Diffusion models: how they work and DP application (6:00)

The talk references several experimental outcomes and comparisons, providing insights into the practical performance of the discussed DP synthetic data generation methods.

For DP synthetic images, the speaker highlights the general state of the art. Examples from the celebrities dataset generated by DP Diffusion models, including those employing advanced techniques like DP-PROMISE, are shown to still have a "whimsical" quality. This indicates that despite significant technical advancements, achieving photo-realistic, high-fidelity images under strong DP guarantees remains an open challenge. The quality produced by PATE-style algorithms for images is described as even lower, with MNIST data generated by such methods being rudimentary. This underscores the difficulty in applying privacy mechanisms to highly complex and continuous data modalities.

In terms of evaluation metrics for images, the talk mentions Inception Score and FID (Frechet Inception Distance), both commonly used to assess the quality and diversity of generated images. MAUVE is also cited as being highly correlated with FID, offering another reliable proxy metric using appropriate embedding models.

A comparative table (intended for a forthcoming paper) is referenced, offering pragmatic guidance:

• If enough data and compute are available, DP fine-tuning (especially of Diffusion models) is recommended. "Enough data" for DP fine-tuning is noted to be "thousands of images."
• If very low epsilons are required, or if the dataset is small ("couple of hundred or a thousand images"), and the private data distribution is similar to the foundational model's pre-training data, then Private Evolution (PE) is a viable option. PE's ability to achieve low epsilon values (e.g., $\epsilon=1$ or $5$) is a significant advantage.

For DP synthetic tabular data, the discussion touches upon the performance of LLM-based methods against workload-based methods.

• The two-stage DP version of GReaT, with its specialized losses (weighted cross-entropy and numerical understanding loss), showed promising results when compared to a workload-based method like REP.
• A concurrent work by Fonyo, employing a similar two-stage LLM approach, was compared against MST and AIM. In this comparison, AIM, a sophisticated workload-based method, "outperformed them by a lot." This highlights that while LLMs bring semantic understanding, workload-based methods remain highly effective, especially when the goal is to precisely match marginal distributions, which they are explicitly optimized for.

The primary evaluation metrics for tabular data typically involve distributional metrics that measure the difference between the marginals of the real and synthetic data. The speaker emphasizes that "workload-based methods are optimized essentially for matching the marginals," which explains their strong performance on these fidelity metrics. While specific hardware or framework versions are not detailed in the transcript, the mention of DP-SGD, LoRA, and complex models like Diffusion and LLMs implies the need for substantial GPU or TPU resources for training and inference.

Overall, the experimental results presented, though qualitative for images and comparative for tabular, clearly illustrate the ongoing challenges in balancing utility and privacy, the benefits of leveraging pre-trained models, and the distinct strengths of different methodological approaches for varying data types and objectives.

Practical Implications

▶ Watch: Improving DP diffusion models and current quality (8:00)

The insights shared by Natalia Ponomareva have significant practical implications for practitioners, infrastructure teams, model builders, and deployers working with sensitive data. The choice of method for generating DP synthetic data is not one-size-fits-all and depends heavily on the specific data modality, available resources, and the desired balance between utility and privacy.

For image data, the message is clear: generating high-quality DP synthetic images remains extremely challenging. Model builders and deployers should set realistic expectations regarding the visual fidelity of DP-generated images, especially under stringent privacy budgets.

• DP fine-tuning of Diffusion models, particularly with public pre-training and techniques like LoRA, is the recommended approach if large private datasets (thousands of images) and substantial computational resources are available. This strategy generally yields the best quality currently achievable for complex image synthesis.
• For scenarios with smaller private datasets (hundreds to a thousand images) or when very low epsilon values (high privacy) are critical, Private Evolution (PE) offers a compelling alternative. However, practitioners must ensure that their private data distribution aligns reasonably well with the distribution of the foundational model used by PE. If the foundational model was trained on dogs and the private data contains cats, PE might generate dogs that look like cats, which might not be useful.
• Infrastructure teams need to be prepared for the computational demands of DP training, which is significantly more expensive than non-DP training due to gradient clipping, noise addition, and often larger batch sizes or more training steps to compensate for noise.
• Defining the privacy unit at the user-level rather than the example-level is a crucial practical consideration for images, requiring careful application-specific analysis to determine what constitutes a "user" and how their contributions are bounded.

For tabular data, the landscape offers more mature and reliable options, alongside emerging deep learning approaches:

• Workload-based methods are highly effective and pragmatic choices for scenarios where preserving specific marginal distributions or predefined query answers is paramount. Practitioners should leverage the rich body of work in this area, selecting methods that fit their computational budget and specific needs for matching statistical properties. These methods are well-suited for traditional data analysis tasks where statistical fidelity is key.
• LLM-based methods, while promising for their ability to leverage semantic understanding from column names and values, are still evolving in the DP context. Model builders exploring these methods should be aware of the need for two-stage training (format compliance + DP fine-tuning) and specialized loss functions. While they might offer a more "end-to-end" solution, they may not yet surpass the statistical accuracy of optimized workload-based methods for marginal preservation.
• Trade-offs and Limitations: Across both modalities, practitioners must continually balance the utility-privacy trade-off. Stronger privacy guarantees (lower epsilon) typically lead to lower data utility and fidelity. The computational cost of DP methods is also a significant factor, often requiring specialized hardware and longer training times. The talk explicitly discourages fabricating details not present in the transcript; similarly, practitioners should avoid overstating the capabilities of DP synthetic data, especially for images, where "whimsical" results are still common.

In essence, successful deployment of DP synthetic data requires a nuanced understanding of the underlying mechanisms, careful selection of appropriate methods based on data characteristics and objectives, and a pragmatic approach to managing the inherent trade-offs between privacy, utility, and computational resources. The field is rapidly advancing, and continuous comparison with established baselines (like workload-based methods for tabular data) is essential for validating new techniques.

Key Takeaways

• DP synthesis for images faces significant challenges: Achieving high-fidelity, photo-realistic DP synthetic images remains an active research area, with current methods often producing "whimsical" results, highlighting the difficulty of injecting privacy noise into complex visual data.
• Public pre-training is a cornerstone for DP deep learning: Leveraging publicly pre-trained models (e.g., encoder-decoders for diffusion, foundational models for Private Evolution) is crucial for improving the quality of DP synthetic data across modalities, as it reduces the amount of noise required during DP fine-tuning.
• Private Evolution offers a practical alternative for image data: For scenarios with smaller datasets or stricter privacy budgets ($\epsilon=1$ or $5$), Private Evolution is a computationally efficient method that leverages powerful foundational models, though it requires the private data distribution to align with the foundational model's training data.
• Workload-based methods are the gold standard for tabular marginals: Traditional workload-based methods (select-measure-estimate paradigm) are highly optimized for preserving specific marginal distributions in tabular data and often outperform newer deep learning approaches when statistical fidelity is the primary goal.
• LLMs show promise for end-to-end tabular generation, but need refinement: While LLM-based methods like two-stage DP-GReaT leverage semantic understanding, they require specialized training techniques (e.g., weighted cross-entropy, numerical understanding loss, LoRA) and still need to consistently match the performance of advanced workload-based methods for marginal preservation.
• Define privacy units and choose metrics wisely: Clearly defining the privacy unit (e.g., user-level for images) and selecting appropriate evaluation metrics (FID/MAUVE for images, distributional metrics for tabular) are fundamental for effectively developing and assessing DP synthetic data generation techniques.

About the Speaker(s)

This segment of the conference talk was delivered by Natalia Ponomareva. She focused on the methods and techniques for generating Differentially Private synthetic images and tabular data. The broader session, titled "DP-fy your DATA: How to (and why) synthesize Differentially Private Synthetic Data," featured several other distinguished speakers. Alex Bie discussed techniques for DP synthetic text, setting the stage for different modalities. Sergei Vassilvitskii provided introductory remarks, alluding to the challenges and importance of the field. Peter Kairouz was slated to discuss practical considerations for creating synthetic data in production, following Natalia's technical deep dive. Together, these speakers represent a collaborative effort to advance the understanding and application of Differential Privacy in synthetic data generation.