Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models

Anshuman Chhabra, Bo Li, Jian Chen, Prasant Mohapatra, Hongfu Liu

International Conference on Machine Learning 2025 · Oral

In the rapidly evolving landscape of deep learning, the quality and relevance of training data are paramount. Models, especially large language models (LLMs), often learn undesirable behaviors or exhibit suboptimal performance due to noisy, mislabeled, or otherwise detrimental samples within their vast training datasets. The talk "Outlier Gradient Analysis: Efficiently Identifying Detrimental Training Samples for Deep Learning Models," presented by Anshuman Chhabra of the University of South Florida, introduces a novel and computationally efficient approach to address this critical challenge. This work, co-authored with Bo Li, Jian Chen, Prasant Mohapatra, and Hongfu Liu, proposes **Outlier Gradient Analysis (OGA)** as a method to pinpoint and remove these problematic samples, thereby enhancing model performance and interpretability.

AI review

OGA proposes identifying detrimental training samples by treating their per-sample loss gradients as outliers in gradient space, then removing them before retraining. The core hypothesis — that harmful samples are few and produce out-of-distribution gradients — is intuitive and the computational motivation is legitimate, but the paper does not deliver on its theoretical framing. The key claim is asserted and illustrated on toy data rather than derived; the 'technical deep dive' describes a pipeline without specifying or analyzing the outlier algorithm used; and the LLM experiments are…