# Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

**Authors:** Leonardo Petrini  
**arXiv:** [2310.16154](https://arxiv.org/abs/2310.16154)  
**Date:** 2023-10-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.16154v1/

## Key points

- Thesis explores theoretical foundations of deep learning
- Focus on relationship between architecture of deep learning models and data structures
- Aims to address efficacy of deep learning algorithms and overcoming curse of dimensionality
- Deep learning learns relevant representations of data by exploiting structure
- Empirical approach combining experimental studies with physics-inspired toy models
- Simplified models help understand complex behaviors in deep learning systems
- Goal is to bridge gap between theory and practice in deep learning

## Summary

This thesis by Leonardo Petrini explores the theoretical foundations of deep learning, with a focus on understanding the relationship between the architecture of deep learning models and the inherent structures within the data they process. The author aims to address questions such as what drives the efficacy of deep learning algorithms and allows them to overcome the curse of dimensionality, which refers to the difficulty of learning functions in high dimensions due to the exponentially increasing need for data points. The author proposes that one reason for the success of deep learning is its ability to learn relevant representations of data by exploiting their structure. To investigate these questions, Petrini takes an empirical approach to deep learning, combining experimental studies with physics-inspired toy models. These simplified models allow for a better understanding and interpretation of complex behaviors observed in deep learning systems. By studying these behaviors, Petrini aims to bridge the gap between theory and practice in deep learning. The advancements in artificial intelligence, particularly in machine learning subfields like natural language processing and computer vision, have largely been attributed to deep learning models. Deep learning surpasses traditional approaches by effectively extracting relevant features from raw data through multiple computational layers. Petrini's research focuses on understanding why deep learning algorithms are so effective and how they overcome challenges posed by high-dimensional data. The author suggests that one key factor is their ability to learn relevant representations of data by leveraging its underlying structure. Different architectures may exploit different aspects of data structures. To explore these ideas, Petrini uses an empirical approach that combines experimental studies with physics-inspired toy models. These simplified models help shed light on complex behaviors observed in deep learning systems and provide insights into their inner workings. The ultimate goal is to bridge the gap between theory and practice in order to enhance our understanding and application of deep learning techniques. In summary, this thesis delves into the theoretical foundations of deep learning by examining how model architecture relates to inherent structures within processed data. It seeks answers regarding what makes deep learning algorithms effective and how they overcome the curse of dimensionality. The research methodology combines empirical studies with physics-inspired toy models to gain insights into the inner workings of deep learning systems, with the aim of bridging the gap between theory and practice in this field.