# Stable ResNet

**Authors:** Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau  
**arXiv:** [2010.12859](https://arxiv.org/abs/2010.12859)  
**Date:** 2020-10-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2010.12859v2/

## Key points

- The paper introduces a new class of ResNet architectures called Stable ResNet.
- Stable ResNet addresses the issues of gradient vanishing and exploding, as well as loss of expressivity in the infinite depth limit.
- Deep ResNet architectures suffer from these issues as the depth becomes large, limiting their effectiveness in solving complex problems.
- Stable ResNet stabilizes the gradient while ensuring expressivity in the infinite depth limit.
- The proposed architecture achieves better performance than existing methods on various image classification tasks and demonstrates its effectiveness in handling deep neural networks with hundreds or thousands of layers.
- The authors provide theoretical analysis and empirical evidence to support their claims about the stability and expressivity of Stable ResNet.

## Summary

The paper titled "Stable ResNet" introduces a new class of ResNet architectures that address the issues of gradient vanishing and exploding, as well as loss of expressivity in the infinite depth limit. While deep ResNet architectures have achieved state-of-the-art performance on many tasks, they suffer from these issues as the depth becomes large. Recent studies have shown that ResNet might lose expressivity as the depth goes to infinity, which limits their effectiveness in solving complex problems. To overcome these limitations, the authors propose Stable ResNet, which stabilizes the gradient while ensuring expressivity in the infinite depth limit. The proposed architecture achieves better performance than existing methods on various image classification tasks and demonstrates its effectiveness in handling deep neural networks with hundreds or thousands of layers. The authors also provide theoretical analysis and empirical evidence to support their claims about the stability and expressivity of Stable ResNet. Overall, this paper presents an important contribution to the field of deep learning by addressing some fundamental challenges associated with deep neural networks and proposing a novel solution that can significantly improve their performance.