Diffusion model

A diffusion model is the kind of generative AI behind most image generators, such as Stable Diffusion, and many video generators. It creates a picture by starting from pure random noise, like TV static, and removing a little of that noise at each step until a clear image emerges. A text description, the prompt, steers each step, so the noise turns into 'a mountain at sunrise' rather than anything else.

The training is the clever part. Take millions of images with captions and gradually ruin each one by adding noise, step by step, until nothing is left but static. Then train a neural network to do the reverse: given a noisy image and its caption, predict what noise was added so it can be subtracted. A model that is good at undoing one small step of damage can, by repeating it dozens of times, build a brand-new image from static. The modern form of this approach was laid out in a 2020 paper on denoising diffusion, and it quickly overtook earlier techniques in image quality.

Running directly on millions of pixels is slow, so most systems use latent diffusion: they first compress images into a much smaller internal representation, do the denoising there, and expand the result into a full image at the end. The same method works for video, audio and even protein design. Because the output depends on the random noise it starts from, the same prompt gives a different picture each time. And because these models learn from huge collections of images from the web, they raise ongoing questions about artists' rights, deepfakes and the stereotypes they reproduce.