Flowing Through Continuous-Time Generative Models: A Clear and Systematic Tour: Flow Through Generative Modeling: A Tutorial

Flowing Through Continuous-Time Generative Models: A Clear and Systematic Tour: Flow Through Generative Modeling: A Tutorial

Speakers: Qiang Liu

Conference: ICML 2025

YouTube: https://slideslive.com/39043356

Overview

This tutorial, presented by Qiang Liu at ICML 2025, offers a comprehensive and systematic exploration of continuous-time generative models, with a particular focus on Rectified Flow (RF). The core problem addressed is turning noise into meaningful data, a fundamental challenge in machine learning with vast applications from text to image and video generation. Liu highlights a significant paradigm shift in generative modeling: from traditional "one-step" models like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) to more recent "iterative process models" such as diffusion models, flow models, and autoregressive models like GPT. This shift is crucial because it decomposes the complex generation task into numerous, simpler steps, distributing the difficulty and leading to higher quality results.

Rectified Flow, as presented, provides a unifying framework for understanding these iterative models, offering both deep theoretical insights and immediate practical applications. It proposes a novel approach to construct and optimize continuous trajectories that transport a simple noise distribution to a complex data distribution. The talk delves into the mathematical underpinnings of this process, its key properties like marginal preservation and transport cost reduction, and its connections to other prominent generative modeling techniques. By systematically dissecting the mechanics of RF, Liu aims to demystify these powerful models and bridge the gap between their theoretical elegance and practical utility, offering a clearer path toward faster and more understandable iterative process models.

Background

▶ Watch: Introduction to generative models and problem statement (0:00)

The evolution of generative modeling has seen a distinct shift. Historically, models like GANs, VAEs, and Normalizing Flows (also known as invertible models) were "one-step" approaches. They treated the learning process as a black box: noise input, neural network processing, and data output. While impactful, these models often struggled with training stability, mode collapse, or limitations in expressiveness for complex, high-dimensional data.

The past five years have witnessed the rise of "iterative process models," which have become the dominant force in achieving state-of-the-art generative quality. These models, including diffusion models, flow models based on Stochastic Differential Equations (SDEs) or Ordinary Differential Equations (ODEs), and autoregressive models like GPT, learn an algorithm or an iterative procedure to transform noise into data. This decomposition of the generation task into many (even infinitely many, in continuous-time models) steps is fundamentally advantageous. It distributes the learning difficulty, making training easier and leading to more powerful models capable of generating significantly higher quality results.

The underlying idea harks back to classical statistics: augmenting data into a latent path connecting noise to data. However, modern approaches differ significantly from traditional methods like Expectation-Maximization (EM). In contemporary iterative models, especially diffusion and flow models, the "augmenting step" (the forward process from data to noise) is often fixed to a simple, known process (e.g., a simple diffusion or interpolation). The primary learning task then becomes imitating the reverse process (from noise to data). This fixation is enabled by the immense capacity of modern neural networks, which can adapt to almost any chosen augmentation process, unlike classical models with limited parameter spaces. A crucial, subtle point is that the learned generative model (P_theta) doesn't need to exactly match the augmented distribution (Q_aug) at every point in the latent space; it only needs to match its marginal distributions over time. This allows the generative model to potentially "improve" upon the initial, non-generative augmentation. This paradigm raises new challenges in the vast design space of algorithms: identifying the best practical and theoretical approaches for these continuous-time generative processes.

Key Findings

▶ Watch: Benefit of iterative models: decomposing generation into steps (2:00)

The central contribution of this talk is the Rectified Flow (RF) framework for building continuous-time generative models. RF provides a systematic way to construct a generative ODE by first defining an interpolation process (Q_aug) that connects noise (X0) and data (X1), and then learning an ODE (P_theta) whose velocity field matches the expected slope of this interpolation.

Key findings and properties of the Rectified Flow include:

1. Rectification Operator: This operator transforms an arbitrary stochastic interpolation process (Q_aug) into a deterministic ODE (P_theta) by learning its conditional expected velocity.
2. Marginal Preserving: The rectification operator inherently preserves the marginal distributions of the original interpolation process. This ensures that the generated samples at any intermediate time step maintain the desired statistical properties. This property is deeply connected to Markovian projection and the continuity equation.
3. Transport Cost Reduction: Applying the rectification procedure iteratively (known as reflow) consistently reduces the transport cost between distributions. This is intuitively explained by the "rewiring" of intersecting trajectories in the ODE, which untangles paths and reduces overall "distance" or cost, as demonstrated by Jensen's inequality. Reflow enhances the "straightness" of trajectories.
4. Straight Transports: The ideal scenario for efficient generation is perfectly straight trajectories. RF aims to achieve these by minimizing the "non-straightness" of the learned ODE. In 1D, a single reflow step is theoretically guaranteed to produce perfectly straight trajectories.
5. L2 Optimal Transport Connection: While RF doesn't directly solve optimal transport problems, it can be extended to achieve L2 optimal transport by constraining the learned velocity field to be a gradient field. This involves implicitly performing a Helmholtz decomposition to remove rotational components from the velocity.
6. Equivariance to Deformations: Rectified Flow exhibits equivariance to deformations, meaning that if two interpolation processes are deformations of each other, their rectified flows will also be deformations of each other. This property explains connections between different interpolation schedules used in models like DDPM and DDIM.
7. Tweedie's Formula for Gaussian Noise: In the special and widely used case where the noise is Gaussian and independent of the data, a powerful connection emerges between the velocity field and the score function (gradient of log-density) via Tweedie's formula. This allows for conversion between velocity-based and score-based models.
8. Singularity Elimination: The talk identifies and provides a solution for singularities that often appear in the velocity or score functions near the boundaries of the time domain (t=0 or t=1). A symmetric parameterization of a shared "slope" m_t can eliminate these singularities, ensuring smooth densities and stable training.
9. ODE-SDE Conversion: Rectified flow models (ODEs) can be converted into SDEs by adding a Langevin dynamics component based on the score function. This addition acts as a "guardrail," steering particles towards high-density regions and enhancing robustness, especially in poorly trained models.
10. Distillation Techniques: The framework encompasses various distillation methods to accelerate inference. This includes distilling the ODE process (e.g., consistency models, using forward/backward/three-point consistency losses) or distilling the interpolation process (matching the generative model's interpolation velocity to a pre-trained one).
11. Reward Alignment (Tilting): RF can be adapted for reward-guided generation by "tilting" the interpolation process. This involves weighting trajectories by a reward function. For Gaussian noise, specific formulas allow for training-free control or variational approaches to align generation with desired rewards.
12. Constraints and Discrete Data: The talk explores how RF can handle data with explicit constraints (e.g., valid 3D points) or discrete domains (e.g., language tokens). Singular forces can be analytically designed to enforce constraints, while the concept of continuous latents in an embedded space offers a promising alternative to discrete jump processes for discrete data generation.

Technical Deep Dive

▶ Watch: Key difference: Fixing the augmenting step (Q_aug) (3:40)

Rectified Flow (RF) Mechanism:

The core idea of RF is to learn a continuous Ordinary Differential Equation (ODE) that maps a simple noise distribution to a complex data distribution. This ODE is defined by its velocity field V(x, t). The process begins by defining an interpolation process X_t that connects a noise sample X_0 (e.g., standard Gaussian) to a data sample X_1 (from the target data distribution). A common choice for this interpolation is a straight line: X_t = (1-t)X_0 + tX_1.

The instantaneous velocity of this interpolation is dX_t/dt = X_1 - X_0. The goal is to train a neural network to approximate the conditional expectation of this velocity: V(x_t, t) = E[dX_t/dt | X_t = x_t]. The training objective is a simple mean squared error loss:

min E[||V(X_t, t) - (X_1 - X_0)||^2]

This loss encourages the learned ODE's velocity field to match the average direction of the interpolating paths passing through x_t at time t. A crucial insight is that this process "rewires" trajectories: since ODEs cannot intersect, forcing an ODE to fit intersecting interpolation paths effectively untangles them. For a single data point, the ideal velocity is V(X_t, t) = (X_1 - X_t) / (1-t), which is singular at t=1. However, for a continuous data distribution, the conditional expectation smooths out these singularities, allowing neural networks to learn the velocity field effectively.

Properties of the Rectified Operator:

The mapping from an interpolation process to its corresponding ODE via this conditional expectation is called the rectify operator. It possesses several fundamental properties:

• Marginal Preserving: The rectified ODE preserves the marginal distributions p_t(x) of the original stochastic interpolation process X_t. This is rigorously shown through the continuity equation, ∂p_t/∂t = -∇ · (p_t V_t), where V_t is the expected velocity. This property is also related to Markovian projection, where a non-Markovian process is projected onto its best Markovian approximation in terms of KL divergence.
• Transport Cost Reduction: For any convex cost function, the transport cost of the rectified ODE is always less than or equal to that of the original interpolation. This is proven using Jensen's inequality. Iterative application of the rectification (called reflow) further reduces transport cost and increases the "straightness" of the trajectories, with the straightness improvement being bounded by the reduction in transport cost. In 1D, a single reflow step can yield perfectly straight trajectories.
• L2 Optimal Transport: While RF naturally reduces transport cost, achieving true L2 optimal transport requires an additional constraint. By forcing the learned velocity field V to be a gradient field (i.e., V = ∇Φ for some scalar function Φ), RF can find the L2 optimal transport. This is achieved through Helmholtz decomposition, which separates any vector field into a curl-free (gradient) component and a divergence-free (rotational) component. Removing the rotational component yields the optimal gradient field.
• Equivariance to Deformations: The rectify operator is equivariant to deformations. If two interpolation processes X_t and X'_t are related by a time-dependent deformation, their rectified flows will be similarly deformed. This explains why different time schedules or affine interpolations (like those in DDPM/DDIM) can be related, and why natural Euler methods (where discretization steps follow the original interpolation curve) are equivalent to transforming to a straight interpolation, applying standard Euler, and transforming back.

Gaussian Independent Noise Special Case:

When the noise X_0 is a standard Gaussian and independent of the data X_1, a powerful simplification occurs. Tweedie's formula establishes a linear relationship between the velocity field V(x_t, t) and the score function ∇log(p_t(x_t)):

V(x_t, t) = (1-t)E[X_1|X_t=x_t] - tE[X_0|X_t=x_t]

∇log(p_t(x_t)) = (E[X_1|X_t=x_t] - x_t) / (1-t) - (x_t - E[X_0|X_t=x_t]) / t

This allows for easy conversion between velocity-based and score-based models. Furthermore, the KL divergence between two marginal distributions rho_t and rho'_t can be expressed as an integral of the differences in their velocities and score functions, providing a theoretical link to maximum likelihood estimation for specific weightings.

A common issue is the singularity of velocity at t=1 and score function at t=0. This can be eliminated by parameterizing a symmetric "slope" m_t such that m_t = (V(x_t, t) + ∇log(p_t(x_t))) / (1-t^2). This symmetric parameterization ensures that the generated distribution has smooth, finite densities by implicitly enforcing the differentiability of the velocity at t=1.

ODE-SDE Conversion:

A trained RF model (an ODE) can be converted into an SDE by adding a Langevin dynamics term: dZ_t = V(Z_t, t)dt + σ(t)dW_t - σ(t)^2/2 ∇log(p_t(Z_t))dt. The Langevin term, σ(t)dW_t - σ(t)^2/2 ∇log(p_t(Z_t))dt, acts as a "guardrail" or "osmosis force," pushing particles towards higher density regions without changing the marginal distributions in expectation. This enhances sampling robustness, especially when the learned velocity field is imperfect, as the score function guides samples towards modes.

Distillation Techniques:

To accelerate generation (e.g., reducing the number of ODE steps), distillation methods are employed. Two main categories exist:

1. ODE Distillation (Consistency Models): Here, a student network directly learns the mapping from X_s to X_t (where s and t are different time steps, t > s) produced by a pre-trained teacher ODE. This involves defining consistency losses:

• Forward Consistency: Enforcing X_t from student matches X_t from teacher's one-step solution from X_s.
• Backward Consistency (Bellman Consistency): Enforcing that flowing from X_t to X_1 with the student yields the same X_1 as the teacher. This is akin to Q-learning.
• Three-point Consistency: Enforcing consistency between three arbitrary time points (t1, t2, t3).

Parameterization choices for the student mapping (e.g., first-order X_t = X_s + (t-s)V(X_s, s) or second-order) and the use of stop gradient are critical design decisions.

1. Interpolation Distillation: This approach trains a generative model G_theta such that its induced interpolation process (e.g., (1-t)X_0 + tG_theta(X_0)) has a velocity field that matches the pre-trained teacher's velocity. This often involves a mini-max optimization, alternating between generating samples, estimating their velocity, and updating G_theta. Losses can be based on integrated KL divergence or Fisher divergence.

Reward Alignment (Tilting):

To guide generation towards desired properties (e.g., higher reward), reward alignment techniques are used. This can be conceptualized as "tilting" the probability distribution. Two main strategies:

1. Tilting Before Rectify: The interpolation process itself is reweighted (tilted) by a reward function R(X_1), and then the rectified flow is derived from this tilted process. The resulting velocity field is a reward-weighted average of the slope. For independent Gaussian coupling, the initial distribution remains unchanged.
2. Tilting After Rectify: First, train the RF model, then apply control or steering mechanisms (e.g., classifier guidance) to the learned ODE/SDE. This is more akin to general reinforcement learning or optimal control problems.

For Gaussian cases, the tilted velocity V_R can be expressed as V_R = V + ∇log(E[R(X_1)|X_t]), potentially allowing for training-free control via Taylor approximations or variational methods.

Constraints and Discrete Data:

RF can be adapted for constrained or discrete data:

• Constraints: For data confined to a specific domain (e.g., manifold, valid molecules), singular forces can be analytically designed and added to the velocity field. These forces ensure that trajectories converge to the desired domain at t=1. A key theoretical result is that bounded drifts do not change the support of an SDE, meaning that a well-designed singular force (which is unbounded at the boundary) can enforce constraints while a bounded neural network learns the probability distribution within that support.
• Discrete Data: For discrete data like language tokens, two main approaches exist:

1. Categorical Diffusion (Jump Processes): The interpolation is a discrete jump process, and the rectification involves minimizing KL divergence of jump processes. This typically leads to absorbing diffusion models. However, these are often limited to single-token jumps and can be viewed as randomized autoregressive models.
2. Continuous Latents: The discrete data is embedded into a continuous space, and RF operates within this continuous latent space. Singular forces or rounding mechanisms then project the final continuous sample back to discrete values. This approach offers greater flexibility, richer tools from continuous flow models, and potential for one-step generation. There's a formal connection showing that continuous rectification can effectively capture discrete jump processes.

Experimental Setup & Results

▶ Watch: Challenges and open questions in generative model design (6:00)

As a tutorial, this talk primarily focuses on presenting a theoretical framework and unifying existing concepts in continuous-time generative models. Therefore, it does not detail specific experimental setups, datasets, baselines, or quantitative headline numbers from new research. Instead, it uses conceptual diagrams and toy examples (e.g., 1D and 2D point clouds) to illustrate the intuition behind Rectified Flow, trajectory rewiring, and the effects of properties like marginal preservation and straightness enhancement. The discussion of large-scale models like Stable Diffusion is conceptual, referring to how these principles apply rather than reporting new benchmark results. The speaker mentions personal experiments with reflow on Flax and anecdotal evidence from other researchers suggesting its promise, but no formal results are presented.

Practical Implications

▶ Watch: Speaker's perspective on the field's depth and impact (7:00)

The Rectified Flow framework carries significant practical implications for practitioners, infrastructure teams, model builders, and deployers of generative AI:

• For Practitioners & Model Builders:
• High-Quality Generation: RF provides a systematic approach to building iterative generative models known for producing state-of-the-art results, similar to diffusion models.
• Faster Inference: The emphasis on straight trajectories and distillation techniques (like consistency models) directly addresses the computational cost of iterative models, enabling generation with fewer steps (potentially even one-step), which is crucial for real-time applications.
• Design Flexibility: RF unifies various design choices, such as the initial interpolation process (straight, cosine, affine), the loss function weighting (velocity prediction, noise prediction, data prediction), and network parameterizations. This provides a clearer landscape for model builders to make informed decisions.
• Guided Generation: The reward alignment mechanisms (tilting) offer a principled way to steer model outputs towards desired attributes, which is vital for controllable generation in applications like content creation or drug discovery (e.g., optimizing for specific molecular properties).
• Handling Constraints: The ability to incorporate singular forces for domain enforcement provides a robust method for generating data that adheres to strict physical or logical constraints, relevant for robotics, 3D modeling, or material science.
• Discrete Data Modeling: The exploration of continuous latents for discrete data (like text) offers a promising alternative to traditional discrete diffusion models, potentially enabling more flexible architectures and faster generation for language models.

• For Infrastructure Teams:
• Scalability for Optimal Transport: The connection between RF and optimal transport, particularly the Helmholtz decomposition for L2 optimal transport, suggests new, potentially more scalable algorithms for solving optimal transport problems in high dimensions.
• Reflow Implementation: While powerful, implementing reflow efficiently on large models might pose engineering challenges related to data simulation and storage, requiring optimized distributed computing and memory management.

• Tradeoffs and Limitations:
• Optimal Weighting and Loss Choice: While RF unifies various loss functions through weighting, the "optimal" weighting or best loss form remains an open empirical and theoretical question.
• Neural Network Generalization Bias: The talk highlights that neural networks generalize in specific, often human-perceptible ways when smoothing singular forces, but the underlying reasons for this inductive bias are unclear.
• Open Questions in Distillation and Alignment: Despite advancements, many open questions persist regarding the optimal design of distillation losses, the choice of stop gradients, and the most effective strategies for reward alignment (e.g., tilting before vs. after rectify, choice of reward approximation).
• Singularities: While solutions like symmetric parameterization exist, managing singularities near time boundaries remains a constant consideration in implementation and theoretical analysis.

Overall, Rectified Flow offers a powerful, systematic, and unifying lens through which to understand, design, and optimize continuous-time generative models, pushing towards faster, higher-quality, and more controllable AI generation.

Key Takeaways

• Rectified Flow (RF) unifies continuous-time generative models: It provides a systematic framework for learning ODEs that transform noise into data, encompassing concepts from diffusion models and flow models.
• Iterative generation decomposes complexity: Modern, high-quality generative models succeed by breaking down the complex noise-to-data mapping into many simpler steps, which RF formalizes through trajectory interpolation and velocity matching.
• Core properties ensure robust learning: The rectification operator guarantees marginal preservation and reduces transport cost, leading to straighter and more efficient generative trajectories, especially through iterative reflow.
• Gaussian noise offers powerful simplifications: For independent Gaussian noise, Tweedie's formula links velocity and score functions, enabling unified loss functions, likelihood calculation, and stable parameterizations to mitigate singularities.
• Advanced techniques enhance performance and control: RF supports distillation for faster inference (e.g., one-step generation), reward alignment for guided generation, and mechanisms to handle constrained or discrete data through singular forces or continuous latent spaces.
• A rich landscape of design choices and open questions: The framework highlights numerous empirical and theoretical questions regarding optimal interpolation choices, loss function weightings, distillation strategies, and the fundamental generalization properties of neural networks in this domain.

About the Speaker(s)

Qiang Liu is the distinguished speaker for this tutorial at ICML 2025. While the transcript does not provide an extensive biography, it is clear that he is a leading researcher in the field of generative models, with a particular focus on continuous-time approaches and the theoretical underpinnings of diffusion and flow models. He mentions his students' significant contributions to the tutorial slides, indicating a vibrant research group. Liu's perspective is deeply rooted in mathematics, drawing parallels between diffusion models and quantum discoveries in physics, which motivates his exploration of the fundamental connections within these areas. His work appears to bridge theoretical advancements with practical, state-of-the-art applications in generative modeling. He also promotes an upcoming NeurIPS workshop on sampling, learning, and diffusion in natural sciences, further showcasing his engagement with the broader research community.