MGD$^3$ : Mode-Guided Dataset Distillation using Diffusion Models

Jeffrey A. Chan-Santiago, praveen tirupattur, Gaurav Kumar Nayak, Gaowen Liu, Mubarak Shah

International Conference on Machine Learning 2025 · Oral

This article delves into MGD$^3$, a novel approach to **Dataset Distillation (DD)** that leverages the power of **diffusion models** with a unique **mode-guided sampling strategy**. Presented by Jeffrey Chan-Santiago and collaborators from UCF, IIT, and Cisco Research, the talk addresses critical challenges in generating compact, high-quality, and diverse synthetic datasets that can effectively substitute much larger original datasets for model training. The core innovation lies in its ability to sample a small dataset that is both diverse and representative, without the need for computationally expensive fine-tuning or the risk of mode collapse often associated with generative models.

AI review

MGD³ is a competent engineering contribution to dataset distillation that combines GMM-based mode identification with guided diffusion sampling to improve diversity in synthetic distilled datasets. The method is well-motivated, the ablations are honest, and the empirical results are real. However, this is fundamentally a system paper with heuristic components rather than a theoretical advance — the 'guidance' mechanism is a standard classifier-free/guided diffusion trick applied in a new context, mode discovery via GMM is classical, and the 'Stop Guidance' finding (stop at timestep 20) is an…