Stop Misunderstanding Diffusion Models

Added: 2026-05-27

[00:00:00]Happily, it turns out that there's a different, but mathematically equivalent, way of understanding what diffusion models are really learning.

[00:00:06]The key will be thinking of diffusion models as learning a time-varying vector field.

[00:00:11]This perspective also leads to a more general approach called flow-based models, which have become very popular recently. If we consider the specific point at the end of this 100-step random walk, in our naive diffusion modeling approach, where we ask our model to denoise images a single step at a time, this is equivalent to giving our model the coordinates of the final 100th point in our walk, and asking our model to predict the coordinates of our point at the 99th step.

[00:00:38]Given enough training points, we expect many diffusion paths to go through this neighborhood.

[00:00:44]And on average, our points will be diffusing away from our starting spiral.

[00:00:49]So, our model can learn to point back towards our spiral. Instead of training the model to remove noise from images one step at a time, the team instead trained the model to predict the total noise added across the entire walk.

[00:01:01]On our plot, this is the vector pointing from our 100th step back to the original starting point of the walk.

[00:01:07]It turns out that we can prove that learning to predict the noise added in the final step of our walk is mathematically equivalent to learning to predict the total noise added divided by the number of steps taken.