Where to Intervene? Benchmarking Fairness-Aware Learning on Differentially Private Synthetic Tabular Data

Vinícius Gabriel Angelozzi (Paper author · Centre Inria de l'Université Grenoble Alpes), Héber Hwang Arcolezi (Paper author · ÉTS Montréal)

Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 3 · Beaver

Overview

In an era increasingly defined by data-driven decision-making and stringent privacy regulations, the generation and use of synthetic data have emerged as a promising approach for sharing data while safeguarding sensitive information. However, this seemingly elegant solution introduces complex challenges, particularly concerning privacy and fairness. This talk, delivered by Vinícius Gabriel Angelozzi, a Master's student at the University Grenoble Alpes, delves into the intricate interplay between differential privacy (DP) and fairness-aware learning when applied to synthetic tabular data. The core problem addressed is the "DP disparate impact," where the noise introduced by differential privacy mechanisms disproportionately harms the utility and fairness for minority groups, potentially amplifying existing biases.

Watch on YouTube · Slides

Visual summary for Where to Intervene? Benchmarking Fairness-Aware Learning on Differentially Private Synthetic Tabular Data by Vinícius Gabriel Angelozzi, Héber Hwang Arcolezi
Visual summary for Where to Intervene? Benchmarking Fairness-Aware Learning on Differentially Private Synthetic Tabular Data by Vinícius Gabriel Angelozzi, Héber Hwang Arcolezi

Key moments

  1. 0:45 Synthetic data can reproduce or amplify bias.
  2. 2:05 DP disparate impact: minority groups suffer more.
  3. 3:20 Three families of fairness intervention mechanisms explained.
  4. 4:25 Introducing DP-Fair benchmark for intervention analysis.
  5. 7:00 Initial results: Fairness correction possible with DP.
  6. 9:10 Main conclusion: Post-processing is the strongest intervention.
  7. 9:40 Future work and open-source DP-Fair framework.

Where to Intervene? Benchmarking Fairness-Aware Learning on Differentially Private Synthetic Tabular Data

Speakers: Vinícius Gabriel Angelozzi (Paper author, Centre Inria de l'Université Grenoble Alpes); Héber Hwang Arcolezi (Paper author, ÉTS Montréal)

Conference: PETS

YouTube: https://www.youtube.com/watch?v=dnJ0K6G4X6c

Overview

In an era increasingly defined by data-driven decision-making and stringent privacy regulations, the generation and use of synthetic data have emerged as a promising approach for sharing data while safeguarding sensitive information. However, this seemingly elegant solution introduces complex challenges, particularly concerning privacy and fairness. This talk, delivered by Vinícius Gabriel Angelozzi, a Master's student at the University Grenoble Alpes, delves into the intricate interplay between differential privacy (DP) and fairness-aware learning when applied to synthetic tabular data. The core problem addressed is the "DP disparate impact," where the noise introduced by differential privacy mechanisms disproportionately harms the utility and fairness for minority groups, potentially amplifying existing biases.

The research presented here introduces DP-Fair, an open-source benchmarking framework designed to systematically evaluate whether existing fairness mechanisms can effectively mitigate the fairness loss induced by differential privacy. Furthermore, it seeks to identify the optimal intervention points—pre-processing, in-processing, or post-processing—within the machine learning pipeline to achieve the best trade-off between utility, privacy, and fairness. This work is critical for practitioners and researchers aiming to deploy privacy-preserving data solutions without inadvertently compromising ethical considerations or exacerbating societal inequalities.

The talk underscores the necessity of a holistic approach to data synthesis, one that not only guarantees formal privacy but also proactively addresses potential biases and disparities. By benchmarking various fairness-aware learning strategies against differentially private synthetic data, Angelozzi and his co-author Héber Hwang Arcolezi provide valuable insights into the practical effectiveness and limitations of current methodologies, paving the way for more robust and ethically sound data sharing practices. Their findings demonstrate that while fairness correction is possible in a DP setting, it is often partial, metric-dependent, and surprisingly, post-processing mechanisms show superior stability and effectiveness.

Background

▶ Watch: Synthetic data can reproduce or amplify bias. (0:45)

The concept of synthetic data has gained significant traction in recent years as a privacy-aware method for data sharing and analysis. Instead of sharing original, sensitive records, organizations can generate new datasets that statistically resemble the original but contain no direct links to individual records. The underlying principle is to learn the statistical distribution of the original data and then sample new, synthetic records from this learned distribution. This approach is widely adopted by industry and recognized by regulators as a way to facilitate data access while limiting exposure to original data.

However, the default generation of synthetic data is not without its perils. Two primary concerns frequently arise:

  1. Privacy Concerns: While synthetic data aims to mask original records, sophisticated attacks can sometimes infer sensitive information about individuals from the synthetic dataset, especially if the synthesis mechanism is not robustly private.
  2. Fairness Concerns: A more insidious problem is the tendency of synthetic data generators to reproduce, or even amplify, biases present in the original data. If the original dataset reflects societal inequities (e.g., underrepresentation of certain demographic groups or biased outcomes), the synthetic data will likely inherit and perpetuate these biases, potentially leading to discriminatory outcomes when used for downstream tasks like training machine learning models.

To address the privacy concern, differential privacy (DP) has emerged as the gold standard, offering formal, mathematical guarantees of privacy. When integrated into synthetic data generators, DP mechanisms introduce carefully calibrated noise during the data synthesis process, ensuring that the presence or absence of any single individual's data record does not significantly alter the output synthetic dataset. This provides strong privacy assurances, making DP-synthetic data suitable for a wide range of analytical and machine learning tasks without directly compromising individual privacy.

Despite its strengths in privacy, the introduction of noise by DP mechanisms can inadvertently create a new challenge for fairness: the DP disparate impact. The noise, while critical for privacy, can distort subtle subgroup structures within the data. This distortion often disproportionately affects minority groups, leading to a loss of utility (e.g., lower predictive performance of models trained on such data) and an amplification of group disparity (e.g., models performing significantly worse for minority groups compared to majority groups). As noted in the talk, while majority groups might experience a minor dip in utility, minority groups can suffer substantial performance degradation, creating a significant fairness gap. Correcting this disparity is the central focus of the research.

To measure and correct such disparities, the literature offers various group fairness metrics and fairness mechanisms. Group fairness metrics quantify the difference in outcomes or performance between different demographic groups. Examples used in this study include the Equal Opportunity Difference (EOD), which measures the difference in true positive rates, and the Statistical Parity Difference (SPD), which measures the difference in positive prediction rates between groups. These metrics help identify if a model is treating groups equitably.

Fairness mechanisms, on the other hand, are algorithmic interventions designed to mitigate bias. They typically fall into three broad categories:

  • Preprocessing Mechanisms: These act directly on the training data before a model is trained. They aim to alter the data distribution to ensure fairness (e.g., re-sampling, re-weighting, or transforming attributes).
  • In-processing Mechanisms: These modify the learning algorithm itself or its objective function during the training process. They integrate fairness constraints directly into the model optimization (e.g., adding a fairness regularizer to the loss function).
  • Post-processing Mechanisms: These operate after a model has been trained, acting directly on its predictions. They adjust the model's outputs to ensure fairness without altering the model or the training data (e.g., adjusting prediction thresholds).

Given the known problem of DP disparate impact, the fundamental questions guiding this research are: Can these established fairness mechanisms effectively counteract the fairness loss caused by differential privacy? And if so, at which stage of the machine learning pipeline (pre-processing, in-processing, or post-processing) is intervention most effective in achieving an optimal balance between utility, privacy, and fairness?

Key Findings

▶ Watch: Three families of fairness intervention mechanisms explained. (3:20)

The research presented by Vinícius Gabriel Angelozzi and Héber Hwang Arcolezi sought to answer two critical questions: first, whether existing fairness mechanisms can correct the fairness loss introduced by differential privacy, and second, where in the machine learning pipeline (pre-processing, in-processing, or post-processing) is the optimal point of intervention. Their findings, derived from extensive benchmarking using the DP-Fair framework, provide crucial insights into these complex interactions.

The overarching conclusion is that, yes, it is indeed possible to partially correct fairness even when operating under a differentially private setting. The experiments consistently demonstrated that applying fairness mechanisms, even after the introduction of DP noise, could move the model's performance closer to a fair state. However, this correction is not absolute; it is often partial and critically metric-dependent. Correcting for one fairness metric (e.g., Equal Opportunity Difference) does not automatically guarantee correction for another (e.g., Statistical Parity Difference). This highlights the need for careful selection of fairness metrics aligned with the specific ethical goals of the application.

A particularly significant finding was the surprising efficacy and stability of post-processing mechanisms. Contrary to what might intuitively be expected—that addressing bias earlier in the pipeline would be more effective—the post-processing stage emerged as the strongest and most stable option for intervening to achieve fairness. Among the post-processing mechanisms tested, ROC (Receiver Operating Characteristic) threshold adjustment and Equal Odds were identified as the most stable and effective choices, providing robust fairness correction while maintaining utility as much as possible within the bounds imposed by differential privacy. This suggests that even with a model trained on noisy, potentially biased DP synthetic data, strategic adjustments to its predictions can yield substantial fairness improvements.

The consistency of these findings across different machine learning classifiers—including robust models like XGBoost, as well as Logistic Regression and Random Forest—further strengthens the conclusions. The observed trend of fairness mechanisms moving outcomes towards a fairer state (closer to zero on disparity metrics) was not an artifact of a single, highly robust classifier but a generalizable pattern. While there was an inherent loss of utility due to the differential privacy mechanisms, the fairness interventions were still able to operate effectively within these constraints, demonstrating that the trade-off between privacy and fairness can be managed, albeit with careful consideration.

In summary, the key findings establish that:

  • Fairness mechanisms can indeed mitigate the disparate impact caused by differential privacy on synthetic data.
  • This mitigation is typically partial and highly dependent on the chosen fairness metric.
  • Post-processing mechanisms, particularly those like ROC threshold adjustment and Equal Odds, offer the most stable and effective intervention point for fairness correction in DP settings.
  • These results hold true across a variety of machine learning models, indicating a generalizable principle rather than a model-specific anomaly.

Technical Deep Dive

▶ Watch: Introducing DP-Fair benchmark for intervention analysis. (4:25)

To rigorously investigate the interplay between differential privacy and fairness, the researchers developed DP-Fair, an open-source benchmarking framework. This framework was designed to systematically test various configurations of data, DP synthesizers, fairness mechanisms, and machine learning models, allowing for the isolation and measurement of their individual and combined impacts on utility and fairness metrics. The modular design of DP-Fair enables extensive experimentation and comparison.

The core of the DP-Fair framework is structured around several baselines, each designed to isolate specific effects:

  • Baseline: This represents the control group. It involves training a machine learning model directly on the original, untouched data without any privacy mechanisms or fairness interventions. Metrics collected from this baseline serve as the ideal (but often impractical) upper bound for utility and the inherent fairness/bias present in the original data.
  • DP Only: In this configuration, the original data undergoes a differentially private synthesis process, and the model is then trained on this DP-synthetic data. No fairness mechanisms are applied. This baseline is crucial for quantifying the direct impact of differential privacy on both utility and, more importantly, on fairness metrics, revealing the extent of the DP disparate impact.
  • Fair Only: Here, fairness mechanisms are applied to the original, non-private data before or during model training. No differential privacy is involved. This baseline helps understand the inherent effectiveness of fairness mechanisms in an ideal (non-private) scenario, serving as a benchmark for how much fairness can be achieved without privacy constraints.
  • DP-Fair: This is the primary experimental setup, where both differential privacy and fairness mechanisms are applied. The data is first made differentially private (via synthesis), and then one or more fairness mechanisms are applied to the DP-synthetic data or the model trained on it. This configuration directly addresses the central research questions, allowing for the evaluation of how fairness mechanisms perform in a privacy-preserving context.

The experimental setup within DP-Fair was comprehensive:

  • Data Sets: The benchmark utilized three real-world datasets to ensure practical relevance and six synthetic datasets where the degree and nature of bias could be precisely controlled. This combination allowed for both realistic evaluation and controlled study of bias amplification.
  • Machine Learning Classifiers: Three distinct types of classifiers, each representing a different family, were employed: Logistic Regression (a linear model), Random Forest (an ensemble tree-based model), and XGBoost (a gradient boosting framework known for its robustness). All classifiers were used with their default settings to ensure a fair comparison and avoid tuning biases.
  • Privacy Budgets: A range of 12 different privacy budgets (epsilon values) were tested. These budgets spanned various privacy regimes, from very strict (small epsilon) to less strict (larger epsilon), allowing for an analysis of how privacy-utility-fairness trade-offs shift with varying privacy guarantees.
  • Random Seeds: To ensure statistical robustness and account for variability, 20 different random seeds were used for each experiment run.
  • DP Synthesizers: Two marginal-based differentially private synthetic data generators were chosen: AIM and MST. These synthesizers are representative of common approaches for generating private synthetic tabular data. The talk primarily focused on results obtained with the AIM synthesizer and the Adult dataset due to time constraints, but the full paper includes broader results.
  • Fairness Mechanisms: A total of 8 different fairness mechanisms were integrated and tested across the three intervention stages:
  • Pre-processing (3 mechanisms): These included techniques like re-sampling or re-weighting the data to balance protected groups.
  • In-processing (2 mechanisms): These involved modifying the training objective, for instance, by adding fairness regularization terms.
  • Post-processing (3 mechanisms): These focused on adjusting model predictions, with ROC (Receiver Operating Characteristic) threshold adjustment and Equal Odds post-processing emerging as particularly effective.
  • Ablation Studies: Two ablation studies were conducted to confirm the robustness of the design choices and ensure that the observed effects were genuinely attributable to the intended variables.

The evaluation primarily relied on two types of metrics:

  • Utility Metric: Accuracy was used as the primary measure of predictive performance. Higher accuracy indicates better utility.
  • Fairness Metrics: The talk specifically mentioned Equal Opportunity Difference (EOD) and Statistical Parity Difference (SPD). Both quantify group disparity, with values closer to zero indicating greater fairness. EOD measures the difference in true positive rates between protected and unprotected groups, while SPD measures the difference in positive prediction rates.

The results were visualized using plots where the Y-axis represented accuracy (utility) and the X-axis represented fairness metrics (EOD, SPD). The ideal outcome on these graphs would be a point with high accuracy and a fairness metric value close to zero. The size of the markers on the plots was used to indicate the strictness of the privacy regime, with smaller markers representing stricter privacy budgets (smaller epsilon). The visual analysis consistently showed that while DP-only configurations (X markers) often resulted in high disparity, the DP-Fair settings (represented by other markers) consistently moved towards the zero point on the fairness axis, indicating successful, albeit partial, correction of bias. This robust trend, observed across various classifiers, underscored the viability of applying fairness mechanisms in a differentially private context.

Demo / Proof of Concept

▶ Watch: Main conclusion: Post-processing is the strongest intervention. (9:10)

While the conference talk itself did not feature a live, interactive demonstration, the research actively presents a robust proof of concept in the form of the DP-Fair benchmarking framework. This is not merely a theoretical construct but a fully developed, open-source software tool that allows researchers and practitioners to replicate and extend the experiments presented.

The DP-Fair framework serves as the central artifact of this work, embodying the methodologies and findings discussed. Its open-source nature means that the community can freely access, use, and contribute to it, fostering transparency and reproducibility in research concerning privacy and fairness. The project's availability is highlighted by the speaker, who mentions that the full benchmark is released open source and can be easily installed via pip, the standard package installer for Python. This ease of access significantly lowers the barrier for others to engage with the research, conduct their own experiments, and validate the presented results.

By making the framework publicly available, the authors effectively provide a "demonstration" of their work's practical utility. Users can:

  • Replicate the experiments: Verify the claims regarding the effectiveness of different fairness mechanisms in DP settings.
  • Extend the research: Test new DP synthesizers, novel fairness algorithms, different datasets, or alternative machine learning models within the established framework.
  • Isolate impacts: Utilize the baseline configurations (Baseline, DP Only, Fair Only, DP-Fair) to systematically analyze the individual and combined effects of privacy and fairness interventions.
  • Evaluate trade-offs: Explore the complex relationships between utility, privacy budget, and various fairness metrics for their specific use cases.

The DP-Fair framework, therefore, acts as a powerful tool for ongoing research and development in the intersection of privacy-preserving machine learning and algorithmic fairness. It moves beyond theoretical discussions by providing a concrete, functional platform for empirical investigation, which is arguably more impactful than a one-off live demo for a technical audience. The mention of a QR code leading to the full paper further reinforces the commitment to accessible and verifiable research.

Defensive Implications

▶ Watch: Future work and open-source DP-Fair framework. (9:40)

The findings from the DP-Fair benchmark carry significant implications for organizations and data scientists working with sensitive data, particularly those adopting differentially private synthetic data. The core message is a crucial warning: the application of differential privacy, while essential for protecting individual privacy, does not inherently guarantee fairness and can, in fact, exacerbate existing biases or create new ones, leading to a DP disparate impact. Defenders must proactively address this.

Here are the key defensive implications:

  1. Do Not Assume DP Preserves Fairness: Organizations should dispel the notion that formal privacy guarantees (like differential privacy) automatically translate into fair outcomes. The noise introduced by DP mechanisms can disproportionately affect minority groups, leading to lower utility and amplified disparities. This necessitates a separate and dedicated effort towards fairness.
  2. Integrate Fairness-Aware Mechanisms into the Pipeline: It is imperative to incorporate fairness mechanisms as a standard part of any data processing and model training pipeline that uses differentially private synthetic data. Relying solely on DP for privacy is insufficient for ethical AI development.
  3. Prioritize Post-Processing for Robustness: The research strongly suggests that post-processing mechanisms, such as ROC threshold adjustment and Equal Odds post-processing, are surprisingly effective, stable, and robust interventions for fairness correction in DP settings. Defenders should consider these methods as primary candidates when designing their fairness strategies, especially when concerned about maintaining utility while correcting for bias. Their stability across different classifiers makes them a reliable choice.
  4. Benchmark and Evaluate Rigorously: Organizations must adopt a rigorous benchmarking approach, similar to the DP-Fair framework, to evaluate the fairness of their models trained on DP-synthetic data. This involves:
  • Using multiple fairness metrics: Reliance on a single metric is insufficient, as fairness correction is metric-dependent. Metrics like Equal Opportunity Difference (EOD) and Statistical Parity Difference (SPD) should be routinely calculated and monitored for all protected groups.
  • Testing across varying privacy budgets: Understanding how different privacy budgets (epsilon values) affect the utility-fairness trade-off is crucial for making informed decisions about privacy parameter settings.
  • Evaluating different intervention points: Experimenting with pre-processing, in-processing, and post-processing techniques will help identify the most effective strategies for specific datasets and use cases.
  1. Be Aware of Utility-Fairness Trade-offs: While fairness can be partially corrected, it often comes with a bound on utility loss imposed by differential privacy. Defenders need to understand and quantify these trade-offs to make informed decisions that align with organizational values and regulatory requirements. This involves careful risk assessment and stakeholder engagement.
  2. Extend Beyond Binary Classification and Single Attributes: The current research focused on binary classification tasks and single protected attributes. Defenders should acknowledge these limitations and extend their internal evaluations to multi-class classification problems and scenarios involving multiple, potentially intersecting, protected attributes, as these can introduce more complex fairness challenges.
  3. Leverage Open-Source Tools: The availability of the DP-Fair framework as an open-source tool is a significant asset. Defenders can leverage this framework to jumpstart their internal evaluations, understand best practices, and contribute to the community's collective knowledge on this critical topic.

In essence, the defensive posture must shift from merely "privacy-preserving" to "privacy-preserving and fairness-aware." This requires a proactive, data-driven strategy that continuously monitors, evaluates, and intervenes to ensure that privacy mechanisms do not inadvertently perpetuate or amplify societal biases in the pursuit of data utility.

Key Takeaways

  • Differential privacy (DP) in synthetic data can exacerbate fairness issues: While DP offers strong privacy guarantees, the noise it introduces can disproportionately affect minority groups, leading to a "DP disparate impact" that amplifies existing biases and reduces utility for these groups.
  • Fairness mechanisms can partially mitigate DP-induced bias: The research demonstrates that applying fairness-aware learning mechanisms can indeed correct for fairness loss in a differentially private setting, moving model outcomes towards greater equity, though this correction is often incomplete.
  • Effectiveness is metric-dependent: The success of fairness interventions is tied to the specific fairness metric being optimized. Correcting for one metric (e.g., Equal Opportunity Difference) does not automatically guarantee fairness across other metrics (e.g., Statistical Parity Difference).
  • Post-processing mechanisms are highly effective and stable: Surprisingly, interventions at the post-processing stage, such as ROC threshold adjustment and Equal Odds post-processing, proved to be the most stable and strongest options for achieving fairness correction, even compared to pre-processing or in-processing methods.
  • The DP-Fair benchmark provides a robust evaluation tool: The open-source DP-Fair framework offers a systematic way to benchmark and evaluate the complex interplay between privacy, utility, and fairness, facilitating reproducible research and aiding practitioners in making informed decisions.
  • A critical trade-off exists between privacy, utility, and fairness: Organizations must be aware that while fairness can be improved, it operates within the bounds of utility loss imposed by differential privacy. Careful consideration of these trade-offs is essential for ethical and effective deployment of synthetic data.

About the Speaker(s)

Vinícius Gabriel Angelozzi is a key author of the paper underlying this talk and delivered the presentation at PETS. At the time of the talk, he was a Master's student at the University Grenoble Alpes in France and was in the process of transitioning to a PhD program, which he was set to begin in September at Inria in the same city. His research focuses on the intersection of privacy and fairness in machine learning, particularly concerning synthetic data.

Héber Hwang Arcolezi is also a paper author and is affiliated with ÉTS Montréal. While not presenting the talk, his contributions were integral to the research and the development of the DP-Fair framework.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

Everyone slaps 'synthetic data' on their pipeline to dodge privacy regulators, completely ignoring that DP noise flattens minority distributions. Angelozzi and Arcolezi actually measured the damage with DP-Fair, proving that post-processing ROC thresholds beats trying to fix the training data beforehand. Real math, counter-intuitive findings, and pip-installable code.

Heather Calloway (CISO) — MUST SEE

This is a critical warning for any enterprise rushing into synthetic data to bypass privacy restrictions. The researchers demonstrate that applying differential privacy can amplify bias, effectively trading a privacy win for a discrimination lawsuit. It is required reading for your AI governance and legal teams.

→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026

All talks from Privacy Enhancing Technologies Symposium (PETS) 2026