# Improved Noise Schedule for Diffusion Training

**Authors:** Tiankai Hang, Shuyang Gu, Xin Geng, Baining Guo  
**arXiv:** [2407.03297](https://arxiv.org/abs/2407.03297)  
**Date:** 2024-07-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2407.03297v2/

## Key points

- Diffusion models are preferred for generating high-fidelity visual outputs in generative AI applications
- Training a single model to predict noise levels accurately across complexities is challenging and resource-intensive
- Researchers have explored strategies like refining loss weighting mechanisms and optimizing model architectures to improve performance
- Authors Hang, Gu, Geng, and Guo introduced a novel approach to refining the noise schedule in diffusion model training
- Their innovation involves using importance sampling techniques on $\log \text{SNR}$ to strategically increase sample frequency around $\log \text{SNR}=0$
- The proposed method aims to enhance training efficiency and accuracy by focusing on the critical transition point between signal dominance and noise dominance
- Empirical evaluations show the superiority of the enhanced noise schedule over traditional cosine schedules
- Benefits of the optimized noise schedule design are demonstrated on benchmark datasets like ImageNet, showing consistent advantages across various prediction targets

## Summary

In the realm of generative AI, diffusion models have emerged as the preferred method for producing high-fidelity visual outputs in diverse applications. However, training a single model to accurately predict noise levels across a spectrum of complexities is a formidable task that requires extensive iterations and substantial computational resources. To address this challenge, researchers have explored various strategies such as refining loss weighting mechanisms and optimizing model architectures to expedite convergence and enhance overall performance. In their recent study titled "Improved Noise Schedule for Diffusion Training," authors Tiankai Hang, Shuyang Gu, Xin Geng, and Baining Guo introduce a novel approach aimed at refining the noise schedule used in training diffusion models. Their key innovation lies in leveraging importance sampling techniques on the logarithm of the Signal-to-Noise ratio ($\log \text{SNR}$) to create a modified noise schedule that strategically increases sample frequency around $\log \text{SNR}=0$. By focusing on this critical transition point between signal dominance and noise dominance, their proposed method aims to improve training efficiency and facilitate more robust and accurate predictions. Through empirical evaluations, the researchers demonstrate the superiority of their enhanced noise schedule compared to traditional cosine schedules. Furthermore, they showcase the benefits of their optimized noise schedule design on benchmark datasets like ImageNet, illustrating its consistent advantages across various prediction targets. These findings not only contribute to ongoing efforts in optimizing diffusion models but also hold promise for advancing more efficient and effective training paradigms within the field of generative AI.