# Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later

**Authors:** Binxu Wang, John J. Vastola  
**arXiv:** [2303.02490](https://arxiv.org/abs/2303.02490)  
**Date:** 2023-03-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.02490v1/

## Key points

- Diffusion generative models convert pure noise into meaningful images
- Generation process involves establishing an outline first, then adding finer details
- Reverse diffusion model proposed to understand the process
- Individual trajectories in diffusion tend to be low-dimensional
- Scene elements with more variation emerge earlier in generation process
- Early perturbations in diffusion model significantly alter image content more frequently than late perturbations
- Behavior of various trained unconditional and conditional diffusion models aligns with predictions
- Theory used to search for latent image manifold and generate interpretable image variations
- Unexpected similarities between GANs and diffusion models noted
- Findings shed light on how diffusion generative models transform noise into meaningful images
- Insights into behavior of trained diffusion models
- Potential avenues for future research and design improvements in both GANs and diffusion models

## Summary

In their paper titled "Diffusion Models Generate Images Like Painters: an Analytical Theory of Outline First, Details Later," Binxu Wang and John J. Vastola explore how diffusion generative models convert pure noise into meaningful images. They argue that the generation process involves first establishing an outline and then gradually adding finer details. To understand this process, they propose a reverse diffusion model that can be represented by dynamics on a high-dimensional landscape filled with Gaussian-like modes. The authors make several predictions based on this model. Firstly, they suggest that individual trajectories in the diffusion process tend to be low-dimensional. Secondly, they propose that scene elements that exhibit more variation within the training data are likely to emerge earlier in the generation process. Lastly, they hypothesize that early perturbations in the diffusion model will significantly alter image content more frequently than late perturbations. To validate their theory, Wang and Vastola analyze the behavior of various trained unconditional and conditional diffusion models, including Stable Diffusion. They find that these models align with their predictions, providing evidence for the proposed analytical theory. Furthermore, the authors utilize their theory to search for the latent image manifold of diffusion models. By doing so, they introduce a new approach to generating interpretable image variations. Interestingly, they also note unexpected similarities between generation processes in GANs (Generative Adversarial Networks) and diffusion models. Overall, this study sheds light on how diffusion generative models transform noise into meaningful images by initially establishing an outline and gradually incorporating finer details. The findings offer insights into the behavior of trained diffusion models and provide a basis for generating interpretable image variations. Additionally, the similarities between GANs and diffusion models suggest potential avenues for future research and design improvements in both fields which could lead to further advances in generative modeling techniques.