Weight initialization based on gradient similarity for versatile machine unlearning

Doun Lee (Paper author · Kookmin University), Jongyun Shin (Paper author · Kookmin University), Jinwoo Bae (Paper author · Kookmin University), Hyunjoon Cho (Paper author · Kookmin University), Jangho Kim (Paper author · Kookmin University)

Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 3 · Caribou

Overview

The proliferation of machine learning models across various domains has brought forth significant challenges related to data privacy and regulatory compliance, particularly the "right to be forgotten." Machine unlearning emerges as a critical technique to address these concerns, aiming to remove the influence of specific data points from a trained model without requiring a full retraining from scratch. This talk, presented by Dong Won Lee from Kookmin University at PETS, delves into a fundamental vulnerability in current machine unlearning methodologies: their susceptibility to adaptive attackers who can exploit subtle information leakages through model properties not targeted by the unlearning process.

Watch on YouTube · Slides

Visual summary for Weight initialization based on gradient similarity for versatile machine unlearning by Doun Lee, Jongyun Shin, Jinwoo Bae, Hyunjoon Cho, Jangho Kim
Visual summary for Weight initialization based on gradient similarity for versatile machine unlearning by Doun Lee, Jongyun Shin, Jinwoo Bae, Hyunjoon Cho, Jangho Kim

Key moments

  1. 0:45 Problem: Unlearning's weakest property leaks information
  2. 1:00 Example: Salon's unlearning leaks entropy information
  3. 2:50 Existing methods overfitted: Salon and Scrob failures
  4. 4:20 Proposed G-OVL algorithm design approximating exact unlearning
  5. 5:20 Metrical OVL: weight selection using gradient similarity
  6. 6:00 G-OVL performance: consistently lower average gap
  7. 7:35 G-OVL versatility across multiple unlearning evaluations
  8. 8:00 Conclusion: Good unlearning generalizes across properties

Weight initialization based on gradient similarity for versatile machine unlearning

Speakers: Doun Lee (Paper author, Kookmin University); Jongyun Shin (Paper author, Kookmin University); Jinwoo Bae (Paper author, Kookmin University); Hyunjoon Cho (Paper author, Kookmin University); Jangho Kim (Paper author, Kookmin University)

Conference: PETS

YouTube: https://www.youtube.com/watch?v=4lMVOhVVyPY

Overview

The proliferation of machine learning models across various domains has brought forth significant challenges related to data privacy and regulatory compliance, particularly the "right to be forgotten." Machine unlearning emerges as a critical technique to address these concerns, aiming to remove the influence of specific data points from a trained model without requiring a full retraining from scratch. This talk, presented by Dong Won Lee from Kookmin University at PETS, delves into a fundamental vulnerability in current machine unlearning methodologies: their susceptibility to adaptive attackers who can exploit subtle information leakages through model properties not targeted by the unlearning process.

The core premise of the research is that existing unlearning algorithms often "overfit" to specific evaluation metrics, making them robust against certain types of privacy attacks while leaving other avenues exposed. This presentation introduces a novel approach, Weight Initialization Based on Gradient Similarity (WIGS), designed to achieve more versatile and robust machine unlearning. By structurally approximating the ideal "exact unlearning" (retraining from scratch) through a two-step process of partial weight initialization and targeted fine-tuning, WIGS aims to provide comprehensive privacy guarantees across multiple model properties, thereby fortifying models against sophisticated privacy inference attacks.

This work is crucial for advancing the practical deployment of machine unlearning in real-world systems, where privacy breaches can have severe legal and ethical consequences. By highlighting the limitations of current evaluation paradigms and proposing a more holistic solution, the research paves the way for more trustworthy and resilient AI systems that can effectively manage and mitigate privacy risks associated with learned data.

Background

▶ Watch: Problem: Unlearning's weakest property leaks information (0:45)

Machine unlearning is a burgeoning field in machine learning security, driven by the necessity to comply with privacy regulations like GDPR and CCPA, which grant individuals the "right to erasure." Conceptually, unlearning involves modifying a pre-trained model to effectively "forget" specific data points (the forget set) while preserving the knowledge gained from the remaining data (the retain set). The gold standard for evaluating unlearning performance is comparing the unlearned model to a gold model, which is trained from scratch solely on the retain set. An ideal unlearned model should be indistinguishable from this gold model in terms of both utility (performance on new data) and privacy (no residual information about the forget set).

However, the talk identifies a critical flaw in current unlearning research and evaluation: methods often target and evaluate only a single model property. For instance, an algorithm might aim to minimize the probability of predicting the true label of a forget set sample (confidence) or reduce the associated loss. While achieving success in these specific metrics, the model might still retain discernible patterns related to the forget set when examined through other properties, such as entropy of its output distribution.

The presentation illustrates this with two prominent examples:

  • Salon: This method uses random label-based fine-tuning, relabeling forget set samples randomly and fine-tuning the model. While Salon effectively reduces the true label probability and loss for the forget set, making it appear similar to the gold model in these aspects, it often fails to raise the output entropy for these samples. The entropy remains low because the model is still minimizing cross-entropy loss, albeit for a random label. This creates a privacy leakage, as an attacker observing low entropy could infer that the data point was part of the original training set, even if the predicted label is wrong. Interestingly, Salon's original research primarily used confidence-based membership inference attacks for evaluation, which it performed well on.
  • Scrob: This algorithm employs a gradient ascent-based method directly manipulating the loss to unlearn. While Scrob shows excellent performance in loss-based membership inference attacks, it exhibits poor results when evaluated against entropy- and confidence-based attacks. Scrob's evaluation in its original research also focused predominantly on loss-based metrics.

This selective success underscores the hypothesis that current unlearning algorithms may be overfitted to certain model properties because their evaluation has been one-sided. An adaptive attacker could exploit these blind spots by querying the model with forget set samples and observing the model property that reveals the most information, thereby circumventing the intended privacy guarantees. To address this, the researchers propose a new aggregate metric, the average gap, which combines the results of membership inference attacks based on loss, entropy, confidence, and the accuracy of the forget set. A lower average gap indicates closer approximation to the gold model across multiple privacy-relevant dimensions.

Key Findings

▶ Watch: Existing methods overfitted: Salon and Scrob failures (2:50)

The central finding of this research is that the privacy guarantees of existing machine unlearning algorithms are often incomplete and vulnerable to adaptive attacks. By focusing on specific model properties during development and evaluation, these methods inadvertently leave other properties exposed, creating pathways for information leakage. The talk demonstrates this quantitatively by showing that prominent unlearning techniques like Salon and Scrob, while performing well on their intended evaluation metrics (e.g., confidence for Salon, loss for Scrob), exhibit "insufficient unlearning" when assessed against a broader spectrum of privacy metrics, particularly entropy and confidence for Scrob, and entropy for Salon. The researchers observed these methods ranking in the bottom 30% for certain membership inference attacks, directly contradicting their perceived efficacy.

To counter this, the paper introduces Weight Initialization Based on Gradient Similarity (WIGS), an algorithm designed for versatile machine unlearning. WIGS successfully addresses the overfitting problem by structurally approximating the ideal "exact unlearning" process (full retraining from scratch). Its key contributions and findings include:

  1. Comprehensive Privacy: WIGS consistently achieves a low "average gap" across multiple model properties (loss, entropy, confidence, and accuracy on the forget set), indicating a significantly more robust and comprehensive unlearning performance compared to state-of-the-art methods. In their experiments, WIGS showed zero instances of insufficient unlearning across the tested membership inference attacks, a stark contrast to other methods.
  2. Structural Approximation of Exact Unlearning: WIGS decomposes exact unlearning into two actionable steps: partial initialization and shorter fine-tuning using the retain set. This approach leverages the benefits of both full re-initialization (catastrophic forgetting) and fine-tuning (utility recovery) in a more efficient manner.
  3. Gradient Similarity for Weight Selection: The novel use of gradient similarity (specifically, Metrical OVL – Overlapping Area of Gaussian distributions of gradients) allows for intelligent selection of weights to be initialized. By identifying weights with high gradient overlap between the forget and retain sets, WIGS targets the "shared pathways" in the model that are likely to encode sensitive information or interfere with unlearning.
  4. Balanced Utility and Privacy: WIGS demonstrates a favorable privacy-utility trade-off. While maintaining competitive test accuracy, especially in random forget scenarios, it ensures that the forget set information is effectively erased across all evaluated metrics. This balance is crucial for practical applications where both model performance and privacy are paramount.
  5. Versatility Across Tasks and Models: The proposed method's effectiveness was validated across diverse datasets (CIFAR-10, ImageNet, SpeechNet), model architectures (ResNet, VGG, VAT), and even complex tasks like image generation using Stable Diffusion, showcasing its broad applicability and robustness.

These findings collectively highlight a paradigm shift in machine unlearning, moving from single-metric optimization to a more holistic, versatile, and privacy-preserving approach.

Technical Deep Dive

▶ Watch: Metrical OVL: weight selection using gradient similarity (5:20)

The proposed Weight Initialization Based on Gradient Similarity (WIGS) algorithm is meticulously designed to overcome the limitations of single-property unlearning by structurally mimicking the ideal, albeit computationally expensive, process of exact unlearning. Exact unlearning involves two conceptual steps: a full initialization of the model parameters, followed by full training using only the retain set. WIGS approximates this through a more efficient, two-stage process: partial initialization and shorter fine-tuning.

The first stage, partial initialization, is the core innovation. Instead of resetting all model weights, WIGS selectively initializes only a subset of weights. The rationale is that a full re-initialization leads to significant loss of learned utility, requiring extensive retraining. By preserving a portion of the weights, the utility can recover faster with shorter fine-tuning. However, this partial preservation must be strategic to ensure catastrophic forgetting of the forget set information. The challenge lies in identifying which weights to initialize to demagnify the influence of the forget set, while preserving others that contribute to the model's overall utility or help magnify the forgetting effect.

WIGS addresses this by leveraging gradient similarity. The intuition is that weights whose gradients are highly similar for both the forget set and the retain set likely represent shared pathways or features within the neural network. These shared pathways could be where the forget set's information is deeply intertwined with the retain set's knowledge, making it difficult to unlearn selectively. Re-initializing these specific weights can effectively sever the connections that encode the forget set's influence without disrupting the entire model.

The specific mechanism for identifying these weights is as follows:

  1. Gradient Statistic Collection: For each weight in the neural network, the algorithm collects the gradient statistics (e.g., mean and variance of gradients) from both the forget set and the train set (or retain set).
  2. Gaussian Distribution Proxy: These gradient statistics are then used to model the gradient distributions as Gaussian distributions. This provides an efficient proxy for comparing the influence of different data subsets on specific weights.
  3. Metrical Overlapping Area (OVL) Calculation: For each weight, the algorithm calculates the overlapping area (OVL) between the Gaussian distributions derived from the forget set gradients and the train/retain set gradients. A larger OVL indicates higher similarity in how a particular weight is affected by both sets.
  4. Top K% Weight Selection: The weights are then ranked based on their OVL values. The algorithm selects the top K% of weights that exhibit the largest overlapping area. These are the weights deemed most critical for selective initialization.
  5. Weight Initialization: The selected top K% weights are then re-initialized (e.g., randomly re-initialized or set to zero). This step aims to "demagnify" or erase the influence of the forget set from these shared pathways. The remaining (100-K)% weights are kept as they are, preserving the majority of the model's learned knowledge for faster utility recovery.

The second stage is shorter fine-tuning using only the retain set. Because only a partial initialization occurred, the model retains significant utility, allowing it to recover its performance on the retain set much faster than a full retraining. Simultaneously, the targeted initialization of shared pathways ensures that the catastrophic forgetting of the forget set is effectively achieved across all model properties. This two-step process structurally approximates the ideal unlearning, balancing the need for complete erasure of specific data with the efficient preservation of general model utility.

The effectiveness of this approach was validated through extensive experiments. For instance, the "high OVL" variant of their method consistently showed a reasonable and consistently lower average gap as training progressed, unlike other methods that exhibited an "uptrend" in the average gap, signifying re-acquisition of forget set information. This demonstrates WIGS's ability to maintain unlearning efficacy throughout the fine-tuning phase.

Demo / Proof of Concept

▶ Watch: G-OVL performance: consistently lower average gap (6:00)

While the talk did not feature a live, interactive demo, the researchers presented compelling experimental results that serve as robust proof of concept for their Weight Initialization Based on Gradient Similarity (WIGS) method. The core of their demonstration revolved around the efficacy of WIGS in achieving versatile unlearning compared to existing techniques across various evaluation metrics and scenarios.

A key aspect of their proof was the introduction and utilization of the average gap metric. This metric aggregates the results of membership inference attacks based on loss, entropy, and confidence, along with the accuracy of the forget set. By measuring the gap between the unlearned model and the gold model across these diverse properties, the average gap provides a comprehensive assessment of unlearning quality. The presented plots clearly showed that as training progressed, other unlearning methods often exhibited an "uptrend" in the average gap, indicating a gradual re-acquisition of information from the forget set. In stark contrast, their "high OVL" (overlapping area) method consistently maintained a reasonable and lower average gap, demonstrating its ability to prevent such information leakage over time.

Further evidence was provided by their Figure 1A, which quantified the "number of insufficient unlearning" instances. This was defined as ranking in the bottom 30% of performance for a given membership inference attack. The results were striking:

  • Salon, while performing well for loss and confidence-based attacks (its original evaluation metrics), showed significant failure for entropy-based membership inference attacks. This directly supported the speaker's initial example of Salon's privacy leakage via entropy.
  • Scrob demonstrated strong performance for loss-based attacks but exhibited poor results for both entropy and confidence-based membership inference attacks, aligning with the researchers' hypothesis of single-metric overfitting.
  • In contrast, the proposed WIGS algorithm (labeled "our proposal" in the figure) showed zero insufficient unlearning across all evaluated membership inference attacks (loss, entropy, and confidence). This unequivocally highlighted its superior versatility and robustness.

The experiments were conducted using standard datasets and architectures, including CIFAR-10 with ResNet, and explored both random forgetting and class forgetting scenarios. WIGS consistently achieved the smallest average gap in both scenarios. For random forgetting, it also demonstrated the best test accuracy, indicating a strong balance between privacy and utility. In the more challenging class forgetting scenario, while there was a slight decrease in test accuracy, this was acknowledged as a natural privacy-utility trade-off inherent in unlearning, but still within acceptable bounds given the enhanced privacy.

The versatility of WIGS was further underscored by additional experiments on diverse models like VGG and VAT, and larger datasets such as ImageNet and SpeechNet. Crucially, the method's applicability extended to complex tasks like image generation using Stable Diffusion and other unlearning scenarios, including selective unlearning. The researchers also evaluated WIGS against four different established unlearning evaluation algorithms, showcasing top performance in tasks like the interclass competition task Julia, the Kaggle emotional challenge, and consistently ranking in the top two for Gaussian poisoning. These comprehensive evaluations solidify WIGS as a highly effective and broadly applicable solution for robust machine unlearning.

Defensive Implications

▶ Watch: Conclusion: Good unlearning generalizes across properties (8:00)

The findings presented in this talk carry significant implications for defenders and organizations deploying machine learning models, particularly those operating under stringent privacy regulations. The core takeaway is a stark warning: the privacy guarantees of machine unlearning are only as strong as their weakest evaluated property. Relying on unlearning methods that merely satisfy a single metric, even if that metric was the focus of the original research, exposes models to sophisticated adaptive attackers.

Defenders must shift their paradigm from single-metric evaluation to a comprehensive and multi-faceted assessment of unlearning efficacy. This means:

  • Demanding Versatile Evaluation: When selecting or implementing unlearning solutions, organizations should require evidence that the method effectively removes information across a broad range of model properties, including but not limited to loss, entropy, and confidence, as measured by various membership inference attacks. The "average gap" metric proposed by the researchers offers a valuable framework for this holistic evaluation.
  • Understanding Adaptive Attack Vectors: Security teams need to internalize that an attacker will not be constrained by the evaluation methods used by developers. If an unlearning algorithm leaves information detectable through, for example, the entropy of model outputs, an adaptive attacker will leverage this specific vulnerability, even if other properties appear secure.
  • Adopting Robust Unlearning Techniques: Methods like Weight Initialization Based on Gradient Similarity (WIGS), which are designed to structurally approximate exact unlearning and ensure comprehensive forgetting across multiple properties, should be prioritized. Integrating such robust techniques can significantly enhance the privacy posture of deployed models.
  • Continuous Monitoring and Re-evaluation: The unlearning landscape is evolving. Defenders should continuously monitor their unlearned models against new and emerging privacy inference attacks. What is considered "sufficient" unlearning today might not be tomorrow.
  • Educating Stakeholders: It is crucial to educate product managers, legal teams, and other non-technical stakeholders about the nuances of machine unlearning and the risks associated with incomplete privacy guarantees. This ensures that privacy-by-design principles are genuinely implemented, rather than merely paying lip service to compliance.
  • Considering the Trade-off: While WIGS demonstrates an excellent privacy-utility balance, defenders must be prepared to accept potential, albeit minor, utility degradation in class-forgetting scenarios. This is a necessary trade-off for achieving robust privacy and should be factored into risk assessments.

In essence, this research serves as a call to action for the security community to elevate the standards of machine unlearning. By understanding and mitigating the risks of single-property overfitting, defenders can build more resilient and trustworthy AI systems that truly honor the right to be forgotten.

Key Takeaways

  • Incomplete Privacy: Current machine unlearning algorithms often "overfit" to specific evaluation metrics, leaving other model properties vulnerable to information leakage and adaptive privacy inference attacks.
  • Adaptive Attacker Threat: An adaptive attacker can exploit these blind spots by querying the model for the property that reveals the most information about the "forgotten" data, circumventing intended privacy guarantees.
  • Comprehensive Evaluation is Crucial: The "average gap" metric, combining loss, entropy, confidence, and forget set accuracy, provides a more holistic and robust way to evaluate unlearning effectiveness across multiple dimensions.
  • WIGS for Versatile Unlearning: The proposed Weight Initialization Based on Gradient Similarity (WIGS) algorithm achieves versatile and comprehensive unlearning by structurally approximating exact unlearning through partial weight initialization and shorter fine-tuning.
  • Gradient Similarity as a Key Mechanism: WIGS leverages Metrical OVL (overlapping area of gradient distributions) to identify and selectively re-initialize "shared pathways" in the model, effectively erasing forget set information while preserving utility.
  • Enhanced Defensive Posture: Defenders must prioritize unlearning solutions that demonstrate robustness across multiple privacy metrics, moving beyond single-metric benchmarks to build truly resilient and compliant AI systems.

About the Speaker(s)

The talk was presented by Dong Won Lee, who is listed as a paper author along with Jongyun Shin, Jinwoo Bae, Hyunjoon Cho, and Jangho Kim. All authors are affiliated with Kookmin University. Their collective research focuses on advancing machine unlearning techniques, particularly addressing the critical challenge of ensuring comprehensive privacy guarantees against adaptive adversaries in the context of machine learning models. Their work aims to push the boundaries of current unlearning methodologies to develop more robust and versatile solutions for data privacy in AI.

Reviews

Dr. Zero (Offensive Security Researcher) — SOLID

Most machine unlearning is just compliance theater where researchers grade their own homework. This PETS paper calls that out, proving how current unlearning methods leak data through output entropy even when their target loss metrics look clean. They propose a clever gradient-overlap technique to surgically nuke shared weights before fine-tuning, bringing us one step closer to unlearning that isn't a joke.

Heather Calloway (CISO) — MUST SEE

This is required reading for any organization building models subject to GDPR or CCPA. The researchers prove that current 'machine unlearning' techniques are largely performative, masking data rather than deleting it and leaving companies legally exposed to privacy extraction.

→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026

All talks from Privacy Enhancing Technologies Symposium (PETS) 2026