# A survey of the Vision Transformers and its CNN-Transformer based Variants

**Authors:** Asifullah Khan, Zunaira Rauf, Anabia Sohail, Abdul Rehman, Hifsa Asif, Aqsa Asif, Umair Farooq  
**arXiv:** [2305.09880](https://arxiv.org/abs/2305.09880)  
**Date:** 2023-05-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.09880v3/

## Key points

- Vision transformers are gaining popularity as an alternative to CNNs in computer vision applications.
- Transformers excel at capturing global relationships in images, but struggle with limited generalization due to their inability to model local correlations.
- Hybrid vision transformers combine the convolution operation and self-attention mechanism to address this limitation.
- These hybrids, also known as CNN-Transformer architectures, effectively exploit both local and global image representations and have shown remarkable performance in vision tasks.
- This survey provides a taxonomy specifically for recent hybrid vision transformer architectures.
- Key features discussed include attention mechanisms, positional embeddings, multi-scale processing, and convolution.
- The survey emphasizes the emerging trend of hybrid vision transformers rather than individual architectures or CNNs alone.
- It showcases the exceptional performance of hybrid vision transformers across various computer vision tasks and sheds light on future directions for this architecture.

## Summary

Vision transformers have gained popularity as a potential alternative to convolutional neural networks (CNNs) in various computer vision applications. These transformers excel at capturing global relationships within images, resulting in high learning capacity. However, they often struggle with limited generalization due to their inability to model local correlations in images. To address this limitation, hybrid vision transformers have emerged by combining the convolution operation and self-attention mechanism. Also known as CNN-Transformer architectures, these hybrids effectively exploit both local and global image representations and have demonstrated remarkable performance in vision tasks. With the growing number of hybrid vision transformers, there is a need for a taxonomy and explanation of these architectures. This survey aims to provide such a taxonomy specifically for recent vision transformer architectures, focusing on the hybrid variants. The survey also discusses key features of these architectures including attention mechanisms, positional embeddings, multi-scale processing, and convolution. What sets this survey apart from previous papers is its emphasis on the emerging trend of hybrid vision transformers rather than individual architectures or CNNs alone. By showcasing the exceptional performance of hybrid vision transformers across various computer vision tasks, this survey sheds light on the future directions of this rapidly evolving architecture. The paper titled "A survey of the Vision Transformers and its CNN-Transformer based Variants" is authored by Asifullah Khan et al., providing a comprehensive overview of vision transformers and their hybrid variants. By highlighting that while vision transformers offer large learning capacity by focusing on global relationships in images they may lack generalization due to limited modeling of local correlation; hybrid vision transformers aim to overcome this limitation by combining convolution operations with self-attention mechanisms. This paper presents a taxonomy specifically for hybrid vision transformer architectures and discusses key features such as attention mechanisms and positional embeddings. Unlike previous surveys that primarily focus on individual architectures or CNNs alone, this survey emphasizes the emerging trend of hybrid vision transformers which can deliver exceptional performance in computer vision tasks; providing insights into the future directions of this rapidly evolving architecture. The paper is 58 pages long and includes 14 figures; falling under Computer Vision (cs.CV).