# Autoregressive Image Generation without Vector Quantization

**Authors:** Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, Kaiming He  
**arXiv:** [2406.11838](https://arxiv.org/abs/2406.11838)  
**Date:** 2024-06-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.11838v2/

## Key points

- Authors challenge the need for vector-quantized tokens in autoregressive models for image generation
- Propose a novel approach using diffusion procedure to model per-token probability distribution in a continuous-valued space
- Introduce Diffusion Loss function as an alternative to categorical cross-entropy loss, eliminating the need for discrete-valued tokenizers
- Method yields strong results across various scenarios without relying on vector quantization
- Enhances efficiency of image generation and allows for faster processing through sequence modeling
- Code for study is openly available at https://github.com/LTH14/mar

## Summary

In their work titled "Autoregressive Image Generation without Vector Quantization," authors Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He challenge the conventional wisdom that autoregressive models for image generation require vector-quantized tokens. They argue that while a discrete-valued space may aid in representing categorical distributions, it is not essential for autoregressive modeling. Instead, the authors propose a novel approach where they model the per-token probability distribution using a diffusion procedure, enabling the application of autoregressive models in a continuous-valued space. By eschewing categorical cross-entropy loss in favor of their newly defined Diffusion Loss function, the researchers eliminate the need for discrete-valued tokenizers. Through extensive evaluation across various scenarios, including standard autoregressive models and generalized masked autoregressive (MAR) variants, they demonstrate that their method yields strong results without relying on vector quantization. This not only enhances the efficiency of image generation but also allows for faster processing due to the advantages of sequence modeling. The authors hope that their innovative approach will inspire further exploration of autoregressive generation in other continuous-valued domains and applications. For those interested in replicating or building upon their findings, the code for this study is openly available at https://github.com/LTH14/mar.