# SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers

**Authors:** Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Yujun Lin, Zhekai Zhang, Muyang Li, Yao Lu, Song Han  
**arXiv:** [2410.10629](https://arxiv.org/abs/2410.10629)  
**Date:** 2024-10-14  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.10629v1/

## Key points

- Authors Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Yujun Lin, Zhekai Zhang, Muyang Li, Yao Lu, and Song Han introduce a novel text-to-image framework called \model.
- \model is capable of efficiently generating high-resolution images up to 4096$\times$4096 resolution with strong text-image alignment at a remarkably fast speed.
- Core designs of \model include deep compression autoencoder for image compression up to 32 times and linear attention in DiT for efficiency at high resolutions without compromising image quality.
- Text encoder in \model replaced with modern decoder-only small Language Model (LLM) for improved image-text alignment through complex human instructions with in-context learning.
- Efficient training and sampling technique called Flow-DPM-Solver reduces sampling steps and accelerates convergence through optimized caption labeling and selection processes.
- Despite being 20 times smaller than modern giant diffusion models like Flux-12B, \model boasts over 100 times faster measured throughput and can generate a high-quality 1024$\times$1024 resolution image in less than one second.
- \model can be run on a standard 16GB laptop GPU, making it accessible for content creators.
- The authors intend to publicly release the code and model associated with \model for further exploration and application within the research community.

## Summary

In their technical report titled "SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers," authors Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Yujun Lin, Zhekai Zhang, Muyang Li, Yao Lu, and Song Han introduce a novel text-to-image framework called \model. This innovative framework is capable of efficiently generating high-resolution images up to 4096$\times$4096 resolution with strong text-image alignment at a remarkably fast speed. Notably, \model can be deployed on a laptop GPU, making it accessible for content creators. The core designs of \model include several key components that contribute to its efficiency and effectiveness. Firstly, the deep compression autoencoder utilized in \model differs from traditional autoencoders by being able to compress images up to 32 times. This significantly reduces the number of latent tokens required for image generation. Additionally, the authors implemented linear attention in place of vanilla attention in DiT (Diffusion Transformer), enhancing efficiency at high resolutions without compromising image quality. Moreover, the text encoder in \model was replaced with a modern decoder-only small Language Model (LLM) instead of T5. This modification was accompanied by the design of complex human instructions with in-context learning to improve image-text alignment. The authors also introduced an efficient training and sampling technique called Flow-DPM-Solver to reduce sampling steps and accelerate convergence through optimized caption labeling and selection processes. As a result of these advancements, demonstrates remarkable competitiveness compared to modern giant diffusion models like Flux-12B. Despite being 20 times smaller than such models, boasts over 100 times faster measured throughput. Furthermore, can be run on a standard 16GB laptop GPU and is capable of generating a high-quality 1024$\times$1024 resolution image in less than one second. Overall,  offers a cost-effective solution for high-resolution image synthesis and content creation due to its impressive performance metrics and accessibility. The authors have announced their intention to publicly release the code and model associated with this groundbreaking framework for further exploration and application within the research community.