# CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers

**Authors:** Ming Ding, Wendi Zheng, Wenyi Hong, Jie Tang  
**arXiv:** [2204.14217](https://arxiv.org/abs/2204.14217)  
**Date:** 2022-04-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2204.14217v2/

## Key points

- Authors address challenges faced by transformer-based text-to-image models:
- Slow generation speed
- Complexity with high-resolution images
- Proposed solution leverages hierarchical transformers and local parallel auto-regressive generation techniques
- Key innovation: Pretraining a 6B-parameter transformer using self-supervised task (CogLM) for subsequent fine-tuning focused on fast super-resolution capabilities
- Resulting system, CogView2, demonstrates remarkable performance in text-to-image generation, competitive with DALL-E-2 model
- Notable advantage of CogView2: Inherent support for interactive text-guided editing on generated images
- Overall, the approach represents a significant advancement in text-to-image generation field by offering improved speed and quality for generating high-resolution visual content from textual inputs

## Summary

In their paper titled "CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers," authors Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang address the challenges faced by transformer-based text-to-image models. These challenges include slow generation speed and complexity when dealing with high-resolution images. To overcome these obstacles, the authors propose a novel solution that leverages hierarchical transformers and local parallel auto-regressive generation techniques. The key innovation introduced in this work is the pretraining of a 6B-parameter transformer using a self-supervised task known as Cross-modal general language model (CogLM). This pretraining process aims to provide a simple yet flexible foundation for the subsequent fine-tuning stage focused on achieving fast super-resolution capabilities. The resulting system, named CogView2, demonstrates remarkable performance in text-to-image generation and showcases competitiveness with the state-of-the-art DALL-E-2 model. One notable advantage of CogView2 is its inherent support for interactive text-guided editing on generated images. This feature enhances user experience and opens up new possibilities for creative applications of text-to-image technology. Overall, the proposed approach represents a significant advancement in the field of text-to-image generation by offering improved speed and quality for generating high-resolution visual content from textual inputs.