# Diffusion model

> An image generator that starts from random noise and cleans it up, step by step.

- Level: Advanced
- Topics: AI
- URL: https://tidbit.legios.com.ar/diffusion-model/
- In Spanish: [Modelo de difusión](https://tidbit.legios.com.ar/es/diffusion-model/)

A **diffusion model** is the kind of generative AI behind most image generators, such as Stable Diffusion, and many video generators. It creates a picture by starting from pure random noise, like TV static, and removing a little of that noise at each step until a clear image emerges. A text description, the prompt, steers each step, so the noise turns into 'a mountain at sunrise' rather than anything else.

The training is the clever part. Take millions of images with captions and gradually ruin each one by adding noise, step by step, until nothing is left but static. Then train a neural network to do the reverse: given a noisy image and its caption, predict what noise was added so it can be subtracted. A model that is good at undoing one small step of damage can, by repeating it dozens of times, build a brand-new image from static. The modern form of this approach was laid out in a 2020 paper on denoising diffusion, and it quickly overtook earlier techniques in image quality.

Running directly on millions of pixels is slow, so most systems use **latent diffusion**: they first compress images into a much smaller internal representation, do the denoising there, and expand the result into a full image at the end. The same method works for video, audio and even protein design. Because the output depends on the random noise it starts from, the same prompt gives a different picture each time. And because these models learn from huge collections of images from the web, they raise ongoing questions about artists' rights, deepfakes and the stereotypes they reproduce.

## An analogy

Like a sculptor who sees a figure in a rough block of marble and chips away a little at a time. Here the block is random static, the chisel is the model, and the prompt tells the sculptor what figure to look for.

## Keep in mind

- Generates images by removing noise in many small steps.
- Trained by adding noise to real images and learning to undo it.
- Latent diffusion works in a compressed space to be faster.
- Same prompt, different starting noise, different image.

## Go deeper

- [Diffusion model on Wikipedia](https://en.wikipedia.org/wiki/Diffusion_model)
- [What are diffusion models? (IBM)](https://www.ibm.com/think/topics/diffusion-models)
- [Denoising Diffusion Probabilistic Models (paper, arXiv)](https://arxiv.org/abs/2006.11239)

## Related terms

- [Generative AI](https://tidbit.legios.com.ar/generative-ai/): AI that creates new text, images, audio or code instead of just sorting things.
- [Prompt](https://tidbit.legios.com.ar/prompt/): Everything you send to an AI model: the question, the instructions, the context.
- [Training and inference](https://tidbit.legios.com.ar/training-and-inference/): Training builds the model once; inference is using it, millions of times a day.
- [Neural network](https://tidbit.legios.com.ar/neural-network/): A pile of tiny math units that learns patterns from examples.
- [Deepfake](https://tidbit.legios.com.ar/deepfake/): A fake video, photo or voice made with AI to look and sound like a real person.
- [Chip](https://tidbit.legios.com.ar/semiconductor-chip/): A sliver of silicon carrying billions of tiny switches that compute and remember.