Normalizing Flows are Capable Generative Models

Shuangfei Zhai, Ruixiang Zhang, Preetum Nakkiran, David Berthelot, Jiatao Gu, Huangjie Zheng, Tianrong Chen, Miguel Angel Bautista Martin, Navdeep Jaitly, Joshua M Susskind

International Conference on Machine Learning 2025 · Oral

This talk, presented by Shuangfei Zhai from Apple's machine learning research team, challenges the long-held notion that **Normalizing Flows (NFs)** are inferior generative models compared to more contemporary approaches like **Diffusion Models** or **Generative Adversarial Networks (GANs)**. Historically, NFs have struggled to produce high-fidelity, diverse samples, often yielding "barely recognizable" images even on standard benchmarks like ImageNet. The core message of this presentation is to convincingly demonstrate that, with the right architectural innovations and training methodologies, normalizing flows can not only achieve state-of-the-art likelihood performance but also generate samples of competitive visual quality.

AI review

Tarflow is a competent and well-executed engineering contribution that rehabilitates normalizing flows as a competitive generative model class. The work earns its benchmark numbers honestly and the combination of transformer-based autoregressive flows with noise augmentation and score-based self-denoising is clean. However, the theoretical grounding for why these techniques work remains largely post-hoc and empirically motivated, the architectural ideas are clearly downstream of existing transformer and diffusion model literature, and the result—while impressive as a number—does not…