Flow Matching

This is what pi0 uses.

Flow matching is a training method used to learn a mapping from a source distribution to a target distribution by approximating the underlying vector field.

The paper:

• https://arxiv.org/pdf/2210.02747

Resources

• https://arxiv.org/pdf/2412.06264 (Flow matching guide and code by meta)
• https://mlg.eng.cam.ac.uk/blog/2024/01/20/flow-matching.html
• https://diffusionflow.github.io/
• https://www.youtube.com/watch?v=DDq_pIfHqLs&ab_channel=Jia-BinHuang
• https://www.youtube.com/watch?v=7cMzfkWFWhI

Flow matching vs. Flow-Based Model?

Flow-based models care about exact probabilities. Flow matching cares about how to move from one distribution to another.

Flow Matching bypasses the need for computing the log-determinant Jacobian or training via score-matching or log-likelihood.

Instead, it trains the flow function $f(x,t)$ to match a target vector field $f^{}(x,t)$ that would push a sample from the base distribution to the target distribution over time.

This makes training more stable.

Flow matching is an alternative way to train CNFs, where you learn the dynamics directly by matching flows instead of optimizing Log Likelihood.

Derivation and my own notes

The flow matching loss in the $\pi 0$ paper is defined as: ${\mathbb{E}}_{(})$ It looks quite

Related

• Normalizing Flow
• Transition Matching