# A Survey of Graph Transformers: Architectures, Theories and Applications

**Authors:** Chaohao Yuan, Kangfei Zhao, Ercan Engin Kuruoglu, Liang Wang, Tingyang Xu, Wenbing Huang, Deli Zhao, Hong Cheng, Yu Rong  
**arXiv:** [2502.16533](https://arxiv.org/abs/2502.16533)  
**Date:** 2025-02-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.16533v1/

## Key points

- Graph Transformers (GTs) have emerged as a powerful tool to overcome limitations of traditional graph neural networks (GNNs).
- Recent advancements in diverse GT architectures enhance explainability and offer practical applications across various domains.
- The paper covers key aspects of GTs including architectural designs, theoretical foundations, and real-world applications.
- GT architectures are categorized based on strategies for processing structural information like graph tokenization, positional encoding, structure-aware attention mechanisms, and model ensemble techniques.
- The authors explore the expressivity of different GT architectures and compare them with advanced graph learning algorithms.
- Practical applications of GTs include molecule analysis, natural language processing tasks, and brain connectivity studies.
- The versatility and effectiveness of GTs in solving complex problems across multiple domains is demonstrated through various use cases.
- Current challenges faced by Graph Transformers are discussed along with potential directions for future research in this rapidly evolving field.

## Summary

In their paper titled "A Survey of Graph Transformers: Architectures, Theories and Applications," authors Chaohao Yuan, Kangfei Zhao, Ercan Engin Kuruoglu, Liang Wang, Tingyang Xu, Wenbing Huang, Deli Zhao, Hong Cheng, and Yu Rong delve into the realm of Graph Transformers (GTs) and their significant impact on modeling graph structures. GTs have emerged as a powerful tool to overcome the limitations of traditional graph neural networks (GNNs), such as over-smoothing and over-squashing. The authors highlight recent advancements in diverse GT architectures that enhance explainability and offer practical applications across various domains. The comprehensive review conducted by the authors covers key aspects of GTs including their architectural designs, theoretical foundations, and real-world applications. They categorize GT architectures based on strategies for processing structural information like graph tokenization, positional encoding, structure-aware attention mechanisms, and model ensemble techniques. From a theoretical standpoint , the authors explore the expressivity of different GT architectures and compare them with advanced graph learning algorithms to uncover underlying connections. Furthermore , the paper provides insights into practical applications where GTs have been successfully utilized. These applications span diverse fields such as molecule analysis , natural language processing tasks , brain connectivity studies . By showcasing these use cases , the authors demonstrate the versatility and effectiveness of GTs in solving complex problems across multiple domains. In conclusion , the authors discuss current challenges faced by Graph Transformers and propose potential directions for future research in this rapidly evolving field. Their work not only sheds light on the capabilities of GTs but also serves as a valuable resource for researchers looking to explore new avenues in graph-based machine learning techniques.