# Deep Learning and Geometric Deep Learning: an introduction for mathematicians and physicists

**Authors:** R. Fioresi, F. Zanchetta  
**arXiv:** [2305.05601](https://arxiv.org/abs/2305.05601)  
**Date:** 2023-05-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.05601v1/

## Key points

- The paper provides an introduction to Deep Learning and Geometric Deep Learning algorithms, with a focus on Graph Neural Networks.
- The dataset used in the study consists of 2708 nodes and 5209 edges representing paper citations.
- Each node is assigned a feature vector consisting of a 1433-dimensional bag-of-words representation of the title of the document and a label assigning it to one of seven distinguished classes.
- The authors build an architecture using Graph Attention Networks (GATs) that achieves an accuracy of 83% on a test set consisting of 1000 nodes.
- The architecture comprises two convolutional layers: ELU(GATConv(1433, 8)) heads = 8 and σ(GATConv(64, 7)) heads = 1.
- The paper covers various topics related to Deep Learning such as supervised classification datasets, training methods for deep learning models including score function and loss function.
- It also delves into Geometric Deep Learning concepts such as graphs and Laplacian on graphs along with heat equation.
- The authors provide some appendices discussing Kullback-Leibler divergence, regression tasks using Multi-layer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs), and Universal Approximation Theorem.

## Summary

This expository paper provides an introduction to the inner workings of Deep Learning and Geometric Deep Learning algorithms, with a focus on Graph Neural Networks. The paper aims to give a brief overview of the key concepts and references for further reading. The dataset used in this study consists of 2708 nodes, each representing a computer science paper, and 5209 edges representing paper citations. Each node is assigned a feature vector consisting of a 1433-dimensional bag-of-words representation of the title of the document and a label assigning it to one of seven distinguished classes: Neural Networks, Case Based, Reinforcement Learning, Probabilistic Methods, Genetic Algorithms, Rule Learning and Theory. The authors build an architecture using Graph Attention Networks (GATs) that achieves an accuracy of 83% on a test set consisting of 1000 nodes. The architecture comprises two convolutional layers: ELU(GATConv(1433, 8)) heads = 8 and σ(GATConv(64, 7)) heads = 1. Here ELU refers to Exponential Linear Unit activation function while σ denotes softmax activation used for final classification. The paper covers various topics related to Deep Learning such as supervised classification datasets, training methods for deep learning models including score function and loss function. It also delves into Geometric Deep Learning concepts such as graphs and Laplacian on graphs along with heat equation. The authors provide some appendices discussing Kullback-Leibler divergence, regression tasks using Multi-layer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs), and Universal Approximation Theorem. Overall, this expository paper serves as an excellent resource for mathematicians and physicists interested in understanding the basics of Deep Learning algorithms with specific emphasis on Graph Neural Networks.