Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Another approach to the problem Not relevant

   M-VAR: Decoupled Scale-wise Autoregressive Modeling for High-Quality Image Generation

   Su-Cheng Ren, Yao-Dong Yu, Nataniel Ruiz et al. · 2024 · arXiv.org · preprint, not peer reviewed · cited 32 times · Not summarized yet

   M-VAR decouples VAR’s next-scale generation into intra-scale and inter-scale modeling to make image generation more computationally efficient.

   Abstract

   There exists recent work in computer vision, named VAR, that proposes a new autoregressive paradigm for image generation. Diverging from the vanilla next-token prediction, VAR structurally reformulates the image generation into a coarse to fine next-scale prediction. In this paper, we show that this scale-wise autoregressive framework can be effectively decoupled into \textit{intra-scale modeling}, which captures local spatial dependencies within each scale, and \textit{inter-scale modeling}, which models cross-scale relationships progressively from coarse-to-fine scales. This decoupling structure allows to rebuild VAR in a more computationally efficient manner. Specifically, for intra-scale…

2. Very close Another approach to the problem Cites this paper Uses this paper in depth Not relevant

   Decoupled Scale-Wise Autoregressive Modeling for Visual Generation

   Su-Cheng Ren, Yao-Dong Yu, Nataniel Ruiz et al. · 2026 · 2026 IEEE/CVF Conference on Computer Vision and Pattern Recognition- FINDINGS Track (CVPRF) · cited 1 time · Not summarized yet

   Decoupled Scale-Wise Autoregressive Modeling separates VAR’s next-scale process into intra-scale and inter-scale components for more efficient visual generation.

   Abstract

   There exists recent work in computer vision, named VAR, that proposes a new autoregressive paradigm for image generation. Diverging from the vanilla next-token prediction, VAR structurally reformulates the image generation into a coarse to fine next-scale prediction. In this paper, we show that this scale-wise autoregressive framework can be effectively decoupled into intra-scale modeling, which captures local spatial dependencies within each scale, and inter-scale modeling, which models cross-scale relationships progressively from coarse-to-fine scales. This decoupling structure allows to rebuild VAR in a more computationally efficient manner. Specifically, for intra-scale modeling - crucia…

3. Very close Another approach to the problem Cites this paper Not relevant

   Visual Implicit Autoregressive Modeling

   Pengfei Jiang, Ji-Xiang Luo, Luxi Lin et al. · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   VIAR replaces VAR’s deep scale-wise stacks with an implicit equilibrium layer to reduce training memory and control inference compute.

   Abstract

   Visual Autoregressive Modeling (VAR) based on next-scale prediction achieves strong generation quality, but their explicit deep stacks fix the amount of computation per scale and inflate memory at high resolutions. We introduce Visual Implicit Autoregressive Modeling (VIAR), a next-scale autoregressive generator that embeds an implicit equilibrium layer between shallow pre/post blocks. The implicit layer is trained with Jacobian-Free Backpropagation, yielding constant training memory, while inference exposes a per-scale iteration knob that enables compute control. On ImageNet 256x256 benchmark, VIAR attains FID 2.16, and sFID 8.07 with only 38.4% parameters of VAR, matching or surpassing str…

4. Very close Same method, another problem Not relevant

   Scale-Wise VAR is Secretly Discrete Diffusion

   Amandeep Kumar, Nithin Gopalakrishnan Nair, Vishal M. Patel · 2025 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet

   Scale-Wise VAR is Secretly Discrete Diffusion analyzes VAR’s next-scale prediction and argues that it is mathematically equivalent to discrete diffusion under a Markovian attention mask.

   Abstract

   Autoregressive (AR) transformers have emerged as a powerful paradigm for visual generation, largely due to their scalability, computational efficiency and unified architecture with language and vision. Among them, next scale prediction Visual Autoregressive Generation (VAR) has recently demonstrated remarkable performance, even surpassing diffusion-based models. In this work, we revisit VAR and uncover a theoretical insight: when equipped with a Markovian attention mask, VAR is mathematically equivalent to a discrete diffusion. We term this reinterpretation as Scalable Visual Refinement with Discrete Diffusion (SRDD), establishing a principled bridge between AR transformers and diffusion mod…

5. Very close Another approach to the problem Not relevant

   Markovian Scale Prediction: A New Era of Visual Autoregressive Generation

   Yu Zhang, Jingyi Liu, Yiwei Shi et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 5 times · Not summarized yet

   Markov-VAR replaces VAR’s full-context scale prediction with a non-full-context Markov process to improve efficiency and performance.

   Abstract

   Visual AutoRegressive modeling (VAR) based on next-scale prediction has revitalized autoregressive visual generation. Although its full-context dependency, i.e., modeling all previous scales for next-scale prediction, facilitates more stable and comprehensive representation learning by leveraging complete information flow, the resulting computational inefficiency and substantial overhead severely hinder VAR's practicality and scalability. This motivates us to develop a new VAR model with better performance and efficiency without full-context dependency. To address this, we reformulate VAR as a non-full-context Markov process, proposing Markov-VAR. It is achieved via Markovian Scale Predictio…

6. Very close Another approach to the problem Not relevant

   HMAR: Efficient Hierarchical Masked Auto-Regressive Image Generation

   Hermann Kumbong, Xian Liu, Tsung-Yi Lin et al. · 2025 · Computer Vision and Pattern Recognition · cited 21 times · Not summarized yet

   HMAR modifies VAR’s hierarchical image generation to address quality loss from parallel within-scale prediction, sequence growth, and fixed sampling schedules.

   Abstract

   Visual Auto-Regressive modeling (VAR) has shown promise in bridging the speed and quality gap between autoregressive image models and diffusion models. VAR reformulates autoregressive modeling by decomposing an image into successive resolution scales. During inference, an image is generated by predicting all the tokens in the next (higher-resolution) scale, conditioned on all tokens in all previous (lower-resolution) scales. However, this formulation suffers from reduced image quality due to the parallel generation of all tokens in a resolution scale; has sequence lengths scaling superlinearly in image resolution; and requires retraining to change the sampling schedule.We introduce Hierarchi…

7. Very close Another approach to the problem Cites this paper Uses this paper in depth Not relevant

   Adversarial Error Correction for Visual Autoregressive Generation

   Ligong Bi, Tao Huang, Jian-Yuan Guo et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 1 time · Not summarized yet

   AID-VAR adds adversarial diagnosis and error correction to pretrained VAR models to limit errors propagating between scales.

   Abstract

   Visual Autoregressive (VAR) models have emerged as a powerful paradigm for image synthesis by performing hierarchical next-scale prediction. However, VAR models are inherently prone to cascading error propagation, where subtle coarse-scale mispredictions are amplified across the hierarchy, ultimately distorting the final synthesis. To mitigate this, we propose AID-VAR, a plug-and-play framework that enhances pre-trained VARs through Adversarially Injected Diagnosis. Instead of a standard passive generation, AID-VAR introduces a proactive error-correction mechanism inspired by the adversarial feedback in GANs. We deploy a discriminator to diagnose fidelity gaps at each scale transition, coupl…

8. Very close Another approach to the problem Not relevant

   FlexVAR: Flexible Visual Autoregressive Modeling without Residual Prediction

   Si-Yu Jiao, Geng-Wei Zhang, Yin-Long Qian et al. · 2025 · Neural Information Processing Systems · cited 25 times · Not summarized yet

   FlexVAR changes VAR’s residual-prediction paradigm, enabling flexible image resolutions and image-to-image generation from low-resolution training.

   Abstract

   This work challenges the residual prediction paradigm in visual autoregressive modeling and presents FlexVAR, a new Flexible Visual AutoRegressive image generation paradigm. FlexVAR facilitates autoregressive learning with ground-truth prediction, enabling each step to independently produce plausible images. This simple, intuitive approach swiftly learns visual distributions and makes the generation process more flexible and adaptable. Trained solely on low-resolution images ($\leq$ 256px), FlexVAR can: (1) Generate images of various resolutions and aspect ratios, even exceeding the resolution of the training images. (2) Support various image-to-image tasks, including image refinement, in/ou…

Prior work it builds on

1. Close Groundwork for this paper Used in depth by this paper Not relevant

   Autoregressive Image Generation using Residual Quantization

   Doyup Lee, Chiheon Kim, Saehoon Kim et al. · 2022 · Computer Vision and Pattern Recognition · cited 1,044 times · Not summarized yet

   Autoregressive Image Generation using Residual Quantization introduces RQ-VAE and RQ-Transformer for high-resolution image generation, which VAR cites.

   Abstract

   For autoregressive (AR) modeling of high-resolution images, vector quantization (VQ) represents an image as a sequence of discrete codes. A short sequence length is important for an AR model to reduce its computational costs to consider long-range interactions of codes. However, we postulate that previous VQ cannot shorten the code sequence and generate high-fidelity images together in terms of the rate-distortion trade-off. In this study, we propose the two-stage framework, which consists of Residual-Quantized VAE (RQ-VAE) and RQ-Transformer, to effectively generate high-resolution images. Given a fixed codebook size, RQ-VAE can precisely approximate a feature map of an image and represent …

2. Close Broader context Used in depth by this paper Not relevant

   Scaling Rectified Flow Transformers for High-Resolution Image Synthesis

   Patrick Esser, Sumith Kulal, Andreas Blattmann et al. · 2024 · Computer Vision and Pattern Recognition · cited 5,151 times · AI summary available

   Scaling Rectified Flow Transformers studies rectified-flow transformers for high-resolution image synthesis, offering a diffusion-based comparison to VAR’s image-generation approach.

   Abstract

   Diffusion models create data from noise by inverting the forward paths of data towards noise and have emerged as a powerful generative modeling technique for high-dimensional, perceptual data such as images and videos. Rectified flow is a recent generative model formulation that connects data and noise in a straight line. Despite its better theoretical properties and conceptual simplicity, it is not yet decisively established as standard practice. In this work, we improve existing noise sampling techniques for training rectified flow models by biasing them towards perceptually relevant scales. Through a large-scale study, we demonstrate the superior performance of this approach compared to e…

3. Close Groundwork for this paper Used in depth by this paper Not relevant

   Scaling Laws for Autoregressive Generative Modeling

   T. Henighan, J. Kaplan, Mor Katz et al. · 2020 · arXiv.org · preprint, not peer reviewed · cited 681 times · Not summarized yet

   Scaling Laws for Autoregressive Generative Modeling measures power-law scaling in autoregressive models, a property VAR also reports for visual generation.

   Abstract

   We identify empirical scaling laws for the cross-entropy loss in four domains: generative image modeling, video modeling, multimodal image$\leftrightarrow$text models, and mathematical problem solving. In all cases autoregressive Transformers smoothly improve in performance as model size and compute budgets increase, following a power-law plus constant scaling law. The optimal model size also depends on the compute budget through a power-law, with exponents that are nearly universal across all data domains. The cross-entropy loss has an information theoretic interpretation as $S($True$) + D_{\mathrm{KL}}($True$||$Model$)$, and the empirical scaling laws suggest a prediction for both the true…

4. Close Groundwork for this paper Not relevant

   Generating Diverse High-Fidelity Images with VQ-VAE-2

   Ali Razavi, A. van den Oord, O. Vinyals · 2019 · Neural Information Processing Systems · cited 2,546 times · Not summarized yet

   Generating Diverse High-Fidelity Images with VQ-VAE-2 uses hierarchical latent representations and autoregressive priors for image generation, which VAR cites.

   Abstract

   We explore the use of Vector Quantized Variational AutoEncoder (VQ-VAE) models for large scale image generation. To this end, we scale and enhance the autoregressive priors used in VQ-VAE to generate synthetic samples of much higher coherence and fidelity than possible before. We use simple feed-forward encoder and decoder networks, making our model an attractive candidate for applications where the encoding and/or decoding speed is critical. Additionally, VQ-VAE requires sampling an autoregressive model only in the compressed latent space, which is an order of magnitude faster than sampling in the pixel space, especially for large images. We demonstrate that a multi-scale hierarchical organ…

5. Related Groundwork for this paper Used in depth by this paper Not relevant

   Scalable Diffusion Models with Transformers

   William Peebles, Saining Xie · 2022 · Computer Vision and Pattern Recognition · cited 7,983 times · AI summary available

   Scalable Diffusion Models with Transformers evaluates transformer-based diffusion image generation, which VAR cites and compares against.

   Abstract

   We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops -- through increased transformer depth/width or increased number of input tokens -- consistently have lower FID. In addition to possessing good scalability properties, our largest DiT-XL/2 models outperform all prior diffusion models on the class-conditional ImageNet 512x512 and 256x256 benchmarks, achieving a…

Papers that cite it

1. Close Another approach to the problem Uses this paper in depth Not relevant

   MEPA: Multi-Scale Representation Alignment for Visual Autoregressive Modeling with Mixture of Experts

   Nuoyan Zhou, Zhi-Jun Tu, Lei Yu et al. · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   MEPA uses scale-aware mixture-of-experts routing to address representation conflicts and error propagation in VAR’s multi-scale image generation.

   Abstract

   Visual AutoRegressive modeling (VAR) has pioneered a coarse-to-fine multi-scale autoregressive generative paradigm, demonstrating strong capabilities in image generation. However, VAR still suffers from inherent deficiencies in multi-scale representation learning. Specifically, lower scales primarily capture global semantics, while higher scales focus on fine-grained details. Employing a shared architecture across scales induces optimization conflicts. Moreover, due to the causal autoregressive process, inaccurate semantics at early scales can propagate and significantly degrade the final output. To address these issues, we introduce a scale-aware token-routed Mixture of Experts (MoE) archit…

2. Close* Same method, another problem Uses this paper in depth Not relevant

   SCALAR++: Efficient Controllable Generation via Scale-wise Visual Autoregressive Learning

   Ryan Xu, Dong-Yang Jin, Shawn Chen et al. · 2026 · International Journal of Computer Vision · cited 1 time · Not summarized yet

   SCALAR++ develops controllable generation using scale-wise visual autoregressive learning, the generation paradigm introduced by VAR.

3. Very close Same method, another problem Not relevant

   ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models

   Fei-Hong Yan, Shao-Yu Liu, Hai-Xuan Wang et al. · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   ExtraVAR adapts VAR’s scale-wise generation to extrapolate beyond its fixed training resolution by remapping stage-specific RoPE frequencies.

   Abstract

   Visual Autoregressive (VAR) models have emerged as a strong alternative to diffusion for image synthesis, yet their fixed training resolution prevents direct generation at higher resolutions. Naively transferring training-free extrapolation methods from LLMs or diffusion models to VAR yields three characteristic failure modes: global repetition, local repetition, and detail degradation. We trace them to a unified band-stage mismatch: VAR generates images in a coarse-to-fine, scale-wise process where each stage is driven by a distinct dominant RoPE frequency band, and each failure mode emerges when the dominant band of a particular stage is disrupted. Building on this insight, we propose Stag…

4. Close Same method, another problem Uses this paper in depth Not relevant

   Training-Free Semantic Correction for Autoregressive Visual Models

   Jun-Hao Chen, Chanyu Zhu, Zhe-Qi Lv et al. · 2026 · arXiv.org · preprint, not peer reviewed · Not summarized yet

   Training-Free Semantic Correction targets errors in next-scale autoregressive visual generation by correcting intermediate states without retraining.

   Abstract

   Autoregressive visual models (AVMs) based on next-scale prediction have emerged as a prominent paradigm for image and video synthesis. However, decomposing the generation process into discrete scales with varying granularities in AVM makes semantic errors difficult to identify and correct, thereby undermining the quality of the final output. Prior efforts to enhance AVM can be categorized into training-based and training-free approaches. Although training-based efforts to enhance AVM generation quality come at substantial computational cost, existing training-free methods neglect intermediate generation states, leaving semantic errors undiagnosed and allowing them to accumulate into the fina…

5. Close Same method, another problem Not relevant

   Knowledge Distillation for Visual Autoregressive Models

   Elia Peruzzo, A. Bhowmik, G. Sautière et al. · 2026 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet

   Knowledge Distillation for Visual Autoregressive Models studies distillation for autoregressive image generators, including the visual AR setting established by VAR.

   Abstract

   Autoregressive (AR) image generation models are highly expressive but computationally intensive, motivating effective model compression. Knowledge distillation (KD) is a natural approach for model compression and has been widely studied in language modeling, yet its behavior in visual AR generation remains underexplored. In this work, we present the first systematic study of distillation strategies for AR image models. Our analysis shows that while standard distillation can yield meaningful gains, recent methods developed for language do not directly transfer to images: long decoding horizons and visual token ambiguity make teacher supervision unreliable especially under student-conditioned …