Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion

A diffusion model is trained to denoise a set of tokens with independent per-token noise levels.

Diffusion forcing = Teacher Forcing + Diffusion Model

Citation

(Chen et al., 2024)

Chen, B., Monso, D. M., Du, Y., Simchowitz, M., Tedrake, R., & Sitzmann, V. (2024). Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion. https://arxiv.org/pdf/2407.01392