# Score-Based Generative Modeling through Stochastic Differential Equations

**Authors:** Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole  
**arXiv:** [2011.13456](https://arxiv.org/abs/2011.13456)  
**Date:** 2020-11-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2011.13456v1/

## Key points

- Groundbreaking study by Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole
- Introduces a novel approach to generative modeling using stochastic differential equations (SDE)
- Core concept involves transforming data distributions into known prior distributions through noise injection and removal
- Forward-time and reverse-time SDEs smoothly transition complex data distributions to priors and vice versa
- Utilizes time-dependent gradient fields or "scores" of perturbed data distributions in the reverse-time SDE
- Neural networks estimate scores for accurate sample generation using numerical SDE solvers
- Framework encapsulates diffusion probabilistic modeling and score-based generative modeling methodologies
- Predictor-corrector framework rectifies errors in discretized reverse-time SDE evolution
- Derives an equivalent neural ordinary differential equation (ODE) for precise likelihood computation and enhanced sampling efficiency
- Enables conditional generation with an unconditional model demonstrated through experiments on class-conditional generation, image inpainting, and colorization
- Record-breaking performance for unconditional image generation on CIFAR-10 dataset with Inception score of 9.89 and Fréchet Inception Distance (FID) of 2.20
- Achieved likelihood at 3.10 bits/dim with high-fidelity $1024 \times 1024$ image generation from a score-based generative model

## Summary

Score-Based Generative Modeling through Stochastic Differential Equations is a groundbreaking study by Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole that introduces a novel approach to generative modeling using stochastic differential equations (SDE). The core concept involves transforming data distributions into known prior distributions through the gradual injection and removal of noise. This process is facilitated by forward-time and reverse-time SDEs that smoothly transition complex data distributions to priors and vice versa. A key highlight of this research is the use of time-dependent gradient fields or "scores" of perturbed data distributions in the reverse-time SDE. By leveraging advancements in score-based generative modeling, neural networks are employed to accurately estimate these scores, enabling the use of numerical SDE solvers for sample generation. The framework presented encapsulates previous methodologies in diffusion probabilistic modeling and score-based generative modeling while also paving the way for innovative sampling procedures. Additionally, the authors introduce a predictor-corrector framework to rectify errors in the evolution of discretized reverse-time SDEs. They also derive an equivalent neural ordinary differential equation (ODE) that samples from the same distribution as the SDE, facilitating precise likelihood computation and enhanced sampling efficiency. Furthermore, this framework enables conditional generation with an unconditional model as demonstrated through experiments on class-conditional generation, image inpainting, and colorization. The research culminates in record-breaking performance for unconditional image generation on CIFAR-10 dataset with an Inception score of 9.89 and Fréchet Inception Distance (FID) of 2.20. The achieved likelihood stands at 3.10 bits/dim while showcasing high-fidelity generation of $1024 \times 1024$ images for the first time from a score-based generative model. In conclusion, Score-Based Generative Modeling through Stochastic Differential Equations presents a cutting-edge approach to generative modeling that not only advances existing methodologies but also opens up new avenues for improved sampling techniques and conditional generation within the realm of artificial intelligence research.