Normalizing Flow

A generative model that turns a simple distribution into a complex one through a chain of invertible transforms, with exact probabilities computable.

A normalizing flow is a class of generative model: starting from a simple distribution such as a standard Gaussian, it passes the sample through a chain of invertible neural-network transforms to arrive at a complex data distribution. Because every step is invertible, the change-of-variables formula from probability theory gives the exact likelihood of a sample, so the model can be trained directly by maximum likelihood; to generate, you just sample noise and run it forward through the network once. The cost is that every layer must be invertible, with an easy-to-compute Jacobian determinant (which measures how much a transform expands or shrinks volume), which restricts the network design. Rezende and Mohamed popularized the idea for variational inference in 2015; representative models include NICE, RealNVP, and Glow. The 2018 continuous normalizing flow wrote the transform as an ordinary differential equation, and flow matching is precisely a simulation-free way of training a continuous normalizing flow — the two share the word 'flow' for a reason.

ExampleGlow (2018) builds a flow model out of coupling layers and invertible 1×1 convolutions, able both to generate realistic face images and to report the exact log-likelihood of any given image.

Also called

Flow-based Generative Model

Related

Generative Model · Flow Matching · Variational Autoencoder · Diffusion Model · Generative Adversarial Network · Velocity Field

Sources

Wikipedia: Flow-based generative model
Variational Inference with Normalizing Flows (arXiv:1505.05770)
Flow Matching for Generative Modeling (arXiv:2210.02747)