Blink of an eye: a simple theory for feature localization in generative models

Marvin Li, Aayush Karan, Sitan Chen

International Conference on Machine Learning 2025 · Oral

Marvin Li, Aayush Karan, and Sitan Chen presented a groundbreaking theoretical framework at ICML 2025 addressing the pervasive phenomenon of "critical windows" in generative AI models. This talk, titled "Blink of an eye: a simple theory for feature localization in generative models," introduces a unified, rigorous, and general theory to explain why and how crucial features of a generation are determined within a very small number of steps during the sampling process, a behavior observed across diverse model architectures like Large Language Models (LLMs) and diffusion models.

AI review

Li, Karan, and Chen present a genuinely unifying theoretical framework for 'critical windows' in generative models — the empirically well-documented phenomenon where a generation's key features are determined within a narrow slice of the sampling trajectory. The core contribution is a clean, dimension-independent theorem parameterized by total variation distance, applicable uniformly to LLMs and diffusion models via the abstraction of stochastic localization samplers. This is real theoretical work: new definitions, stated assumptions, and a result that earns its claimed generality. The…