# On the Approximation Properties of Neural Networks

**Authors:** Jonathan W. Siegel, Jinchao Xu  
**arXiv:** [1904.02311](https://arxiv.org/abs/1904.02311)  
**Date:** 2019-04-04  
**Summary page:** https://www.summarizepaper.com/en/paper/1904.02311v1/

## Key points

- Authors Jonathan W. Siegel and Jinchao Xu present two novel results in their paper titled "On the Approximation Properties of Neural Networks":
- The first result establishes conditions for linear independence of outputs in a two-layer neural network, revealing insights into its representational capacity.
- The second result explores the rate of approximation as the number of neurons increases, providing valuable insights into the network's ability to approximate complex functions.
- Siegel and Xu enhance existing literature by relaxing assumptions on activation functions, making their results more widely applicable.
- Their research demonstrates improved performance in neural network modeling by offering a better rate of approximation compared to previous studies.
- They provide a simplified proof showing that a two-layer neural network can densely cover any compact set when the activation function is not a polynomial, contributing valuable insights for enhanced understanding and utilization of neural networks.

## Summary

In their paper titled "On the Approximation Properties of Neural Networks," authors Jonathan W. Siegel and Jinchao Xu present two novel results that contribute to the understanding of neural networks' approximation capabilities. The first result establishes conditions under which the outputs of neurons in a two-layer neural network are linearly independent functions, shedding light on the network's representational capacity. The second result delves into the rate of approximation of a two-layer neural network as the number of neurons increases, offering insights into how effectively these networks can approximate complex functions. Notably, Siegel and Xu enhance existing literature by relaxing the required assumptions on the activation function, making their results more widely applicable. By providing a better rate of approximation than previous studies, they demonstrate the potential for improved performance in neural network modeling. Additionally, they offer a simplified proof that showcases how the class of functions represented by a two-layer neural network can densely cover any compact set when the activation function is not a polynomial. Overall, this research contributes valuable insights into the approximation properties of neural networks, paving the way for enhanced understanding and utilization of these powerful computational tools in various applications.