# Large-scale Graph Representation Learning of Dynamic Brain Connectome with Transformers

**Authors:** Byung-Hoon Kim, Jungwon Choi, EungGu Yun, Kyungsang Kim, Xiang Li, Juho Lee  
**arXiv:** [2312.14939](https://arxiv.org/abs/2312.14939)  
**Date:** 2023-12-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.14939v1/

## Key points

- Graph Transformers have emerged as a powerful tool for graph representation learning
- They offer advantages over traditional message-passing Graph Neural Networks
- Graph Transformers are gaining interest in learning the representation of brain functional connectivity networks
- Existing studies have overlooked the temporal dynamics of functional connectivity
- The authors propose a novel method using Graph Transformers to learn the representation of dynamic functional connectivity
- Their approach captures position, structure, and temporal information of the functional connectivity graph
- An extensive dataset consisting of over 50,000 resting-state fMRI samples was used to evaluate their method
- Their proposed method outperforms other baselines in gender classification and age regression tasks based on fMRI data
- This study contributes to advancing our understanding of dynamic functional connectivity in the brain

## Summary

In recent years, Graph Transformers have emerged as a powerful tool for graph representation learning in various tasks, offering several advantages over traditional message-passing Graph Neural Networks. One area where the application of Graph Transformers is gaining interest is in learning the representation of brain functional connectivity networks. Existing studies in this field have largely overlooked the temporal dynamics of functional connectivity, which fluctuates over time. To address this limitation, the authors propose a novel method for learning the representation of dynamic functional connectivity using Graph Transformers. The key idea behind their approach is to define a connectome embedding that captures not only the position and structure of the functional connectivity graph but also its temporal information. By leveraging Transformers, they are able to learn the representation of this connectome embedding across time. To evaluate their method, an extensive dataset consisting of over 50,000 resting-state fMRI samples obtained from three different datasets was used. This represents by far the largest number of fMRI data used in similar studies. The experimental results demonstrate that their proposed method outperforms other competitive baselines in gender classification and age regression tasks based on the functional connectivity extracted from fMRI data. This highlights the effectiveness and potential utility of their approach for analyzing brain function and understanding individual differences. Overall, this study contributes to advancing our understanding of dynamic functional connectivity in the brain by introducing a novel method that leverages Graph Transformers to capture temporal dynamics. The findings suggest that considering these dynamics can lead to improved performance in tasks related to gender classification and age regression based on fMRI data.