Graph Neural Networks with Learnable Structural and Positional Representations
AI-generated Key Points
- Graph neural networks (GNNs) are standard learning architectures for graphs with various applications.
- Arbitrary graphs lack canonical positional information of nodes, which decreases the representation power of GNNs.
- Positional Encoding (PE) of nodes has been proposed to address this issue, including Laplacian eigenvectors as possible graph PE.
- Three research directions have been explored: theoretical expressivity of GNNs, graph positional encoding, and Transformer-based GNNs.
- Learnable Structural and Positional Encodings (LSPE) is a new approach that decouples structural and positional representations to improve the performance of GNN models on molecular datasets specifically.
- LSPE outperformed all other models across all tested datasets in terms of mean absolute error (MAE), training time, number of parameters used, and area under the curve (AUC) for training.
Authors: Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, Xavier Bresson
Abstract: Graph neural networks (GNNs) have become the standard learning architectures for graphs. GNNs have been applied to numerous domains ranging from quantum chemistry, recommender systems to knowledge graphs and natural language processing. A major issue with arbitrary graphs is the absence of canonical positional information of nodes, which decreases the representation power of GNNs to distinguish e.g. isomorphic nodes and other graph symmetries. An approach to tackle this issue is to introduce Positional Encoding (PE) of nodes, and inject it into the input layer, like in Transformers. Possible graph PE are Laplacian eigenvectors. In this work, we propose to decouple structural and positional representations to make easy for the network to learn these two essential properties. We introduce a novel generic architecture which we call LSPE (Learnable Structural and Positional Encodings). We investigate several sparse and fully-connected (Transformer-like) GNNs, and observe a performance increase for molecular datasets, from 1.79% up to 64.14% when considering learnable PE for both GNN classes.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Assess the quality of the AI-generated content by voting
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
Similar papers summarized with our AI tools
Navigate through even more similar papers through atree representation
Look for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.