# Simplicial Attention Networks

**Authors:** Christopher Wei Jin Goh, Cristian Bodnar, Pietro Liò  
**arXiv:** [2204.09455](https://arxiv.org/abs/2204.09455)  
**Date:** 2022-04-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2204.09455v1/

## Key points

- Growing interest in using higher-order structures to enhance learning abilities of graph neural networks (GNNs) in combinatorial spaces
- Simplicial Neural Networks (SNNs) have limitations due to strict tie to combinatorial structure of complex
- Simplicial Attention Networks (SAT) proposed as a new type of simplicial network that dynamically weighs interactions between neighboring simplices and can adapt to novel structures
- SAT introduces signed attention mechanism that makes it orientation-equivariant, desirable for models operating on (co)chain complexes
- SAT outperforms existing convolutional SNNs and GNNs in two image and trajectory classification tasks
- Potential lies in ability to model higher-order interactions beyond node-wise connections while being flexible enough to adapt to novel structures
- Proposed method could pave the way for more effective representation learning methods for complex data types such as social networks or biological systems.

## Summary

In recent years, there has been growing interest in exploring how higher-order structures can be used to enhance the learning abilities of graph neural networks (GNNs) in combinatorial spaces. While Simplicial Neural Networks (SNNs) have shown promise in modeling these interactions by performing message passing on simplicial complexes, they are limited by their strict tie to the combinatorial structure of the complex. To address this limitation, a team of researchers led by Christopher Wei Jin Goh, Cristian Bodnar, and Pietro Liò proposed Simplicial Attention Networks (SAT), a new type of simplicial network that dynamically weighs the interactions between neighboring simplices and can readily adapt to novel structures. Leveraging attention mechanisms' success in structured domains, SAT introduces a signed attention mechanism that makes it orientation-equivariant, which is desirable for models operating on (co)chain complexes. The team demonstrated that SAT outperforms existing convolutional SNNs and GNNs in two image and trajectory classification tasks. The proposed method's potential lies in its ability to model higher-order interactions beyond node-wise connections while being flexible enough to adapt to novel structures. This could pave the way for more effective representation learning methods for complex data types such as social networks or biological systems. The paper was accepted at the ICLR 2022 Workshop on Geometrical and Topological Representation Learning.