# Graph Neural Networks with Learnable Structural and Positional Representations

**Authors:** Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson  
**arXiv:** [2110.07875](https://arxiv.org/abs/2110.07875)  
**Date:** 2021-10-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2110.07875v2/

## Key points

- Graph neural networks (GNNs) are standard learning architectures for graphs with various applications.
- Arbitrary graphs lack canonical positional information of nodes, which decreases the representation power of GNNs.
- Positional Encoding (PE) of nodes has been proposed to address this issue, including Laplacian eigenvectors as possible graph PE.
- Three research directions have been explored: theoretical expressivity of GNNs, graph positional encoding, and Transformer-based GNNs.
- Learnable Structural and Positional Encodings (LSPE) is a new approach that decouples structural and positional representations to improve the performance of GNN models on molecular datasets specifically.
- LSPE outperformed all other models across all tested datasets in terms of mean absolute error (MAE), training time, number of parameters used, and area under the curve (AUC) for training.

## Summary

Graph neural networks (GNNs) have become the standard learning architectures for graphs, with applications ranging from quantum chemistry and recommender systems to knowledge graphs and natural language processing. However, a major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish isomorphic nodes and other graph symmetries. To address this issue, researchers have proposed introducing Positional Encoding (PE) of nodes, similar to Transformers. Possible graph PE includes Laplacian eigenvectors. In related work, three research directions have been explored: theoretical expressivity of GNNs, graph positional encoding, and Transformer-based GNNs. Theoretical expressivity studies aim to understand the limitations of GNNs in terms of their ability to learn different graph structures. Graph positional encoding approaches focus on incorporating node positions into GNN models to improve their performance on tasks such as node classification or link prediction. Transformer-based GNNs are inspired by the success of Transformers in natural language processing and aim to incorporate similar attention mechanisms into GNN models. To improve the performance of GNN models on molecular datasets specifically, a new approach called Learnable Structural and Positional Encodings (LSPE) has been proposed. LSPE decouples structural and positional representations to make it easier for the network to learn these two essential properties separately. Researchers investigated several sparse and fully-connected (Transformer-like) GNNs using LSPE and observed a significant increase in performance on molecular datasets when considering learnable PE for both classes of GNNs. The study also compared various existing models such as GraphiT, SAN, PNA with different types of positional encodings - Laplacian eigenvectors or random walk embeddings - on molecular datasets. Results showed that LSPE outperformed all other models across all tested datasets in terms of mean absolute error (MAE), training time, number of parameters used, and area under the curve (AUC) for training. Overall, LSPE provides a promising new approach for improving the performance of GNNs on molecular datasets by incorporating learnable structural and positional encodings.