SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers

AI-generated keywords: SANA text-to-image framework linear diffusion transformers high-resolution image synthesis efficiency

AI-generated Key Points

The license of the paper does not allow us to build upon its content and the key points are generated using the paper metadata rather than the full article.

  • Authors Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Yujun Lin, Zhekai Zhang, Muyang Li, Yao Lu, and Song Han introduce a novel text-to-image framework called \model.
  • \model is capable of efficiently generating high-resolution images up to 4096$\times$4096 resolution with strong text-image alignment at a remarkably fast speed.
  • Core designs of \model include deep compression autoencoder for image compression up to 32 times and linear attention in DiT for efficiency at high resolutions without compromising image quality.
  • Text encoder in \model replaced with modern decoder-only small Language Model (LLM) for improved image-text alignment through complex human instructions with in-context learning.
  • Efficient training and sampling technique called Flow-DPM-Solver reduces sampling steps and accelerates convergence through optimized caption labeling and selection processes.
  • Despite being 20 times smaller than modern giant diffusion models like Flux-12B, \model boasts over 100 times faster measured throughput and can generate a high-quality 1024$\times$1024 resolution image in less than one second.
  • \model can be run on a standard 16GB laptop GPU, making it accessible for content creators.
  • The authors intend to publicly release the code and model associated with \model for further exploration and application within the research community.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Yujun Lin, Zhekai Zhang, Muyang Li, Yao Lu, Song Han

Technical Report
License: CC BY-NC-ND 4.0

Abstract: We introduce \model, a text-to-image framework that can efficiently generate images up to 4096$\times$4096 resolution. \model can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU. Core designs include: (1) Deep compression autoencoder: unlike traditional AEs, which compress images only 8$\times$, we trained an AE that can compress images 32$\times$, effectively reducing the number of latent tokens. (2) Linear DiT: we replace all vanilla attention in DiT with linear attention, which is more efficient at high resolutions without sacrificing quality. (3) Decoder-only text encoder: we replaced T5 with modern decoder-only small LLM as the text encoder and designed complex human instruction with in-context learning to enhance the image-text alignment. (4) Efficient training and sampling: we propose Flow-DPM-Solver to reduce sampling steps, with efficient caption labeling and selection to accelerate convergence. As a result, \model-0.6B is very competitive with modern giant diffusion model (e.g. Flux-12B), being 20 times smaller and 100+ times faster in measured throughput. Moreover, \model-0.6B can be deployed on a 16GB laptop GPU, taking less than 1 second to generate a 1024$\times$1024 resolution image. Sana enables content creation at low cost. Code and model will be publicly released.

Submitted to arXiv on 14 Oct. 2024

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

The license of the paper does not allow us to build upon its content and the AI assistant only knows about the paper metadata rather than the full article.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2410.10629v1

This paper's license doesn't allow us to build upon its content and the summarizing process is here made with the paper's metadata rather than the article.

In their technical report titled "SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers," authors Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Yujun Lin, Zhekai Zhang, Muyang Li, Yao Lu, and Song Han introduce a novel text-to-image framework called \model. This innovative framework is capable of efficiently generating high-resolution images up to 4096$\times$4096 resolution with strong text-image alignment at a remarkably fast speed. Notably, \model can be deployed on a laptop GPU, making it accessible for content creators. The core designs of \model include several key components that contribute to its efficiency and effectiveness. Firstly, the deep compression autoencoder utilized in \model differs from traditional autoencoders by being able to compress images up to 32 times. This significantly reduces the number of latent tokens required for image generation. Additionally, the authors implemented linear attention in place of vanilla attention in DiT (Diffusion Transformer), enhancing efficiency at high resolutions without compromising image quality. Moreover, the text encoder in \model was replaced with a modern decoder-only small Language Model (LLM) instead of T5. This modification was accompanied by the design of complex human instructions with in-context learning to improve image-text alignment. The authors also introduced an efficient training and sampling technique called Flow-DPM-Solver to reduce sampling steps and accelerate convergence through optimized caption labeling and selection processes. As a result of these advancements, demonstrates remarkable competitiveness compared to modern giant diffusion models like Flux-12B. Despite being 20 times smaller than such models, boasts over 100 times faster measured throughput. Furthermore, can be run on a standard 16GB laptop GPU and is capable of generating a high-quality 1024$\times$1024 resolution image in less than one second. Overall, offers a cost-effective solution for high-resolution image synthesis and content creation due to its impressive performance metrics and accessibility. The authors have announced their intention to publicly release the code and model associated with this groundbreaking framework for further exploration and application within the research community.
Created on 31 Oct. 2024

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.