On the Expressive Power of Neural Networks
Auteurs : Jan Holstermann
Résumé scientifique
In 1989 George Cybenko proved in a landmark paper that wide shallow neural networks can approximate arbitrary continuous functions on a compact set. This universal approximation theorem sparked a lot of follow-up research. Shen, Yang and Zhang determined optimal approximation rates for ReLU-networks in $L^p$-norms with $p \in [1,\infty)$. Kidger and Lyons proved a universal approximation theorem for deep narrow ReLU-networks. Telgarsky gave an example of a deep narrow ReLU-network that cannot be approximated by a wide shallow ReLU-network unless it has exponentially many neurons. However, there are even more questions that still remain unresolved. Are there any wide shallow ReLU-networks that cannot be approximated well by deep narrow ReLU-networks? Is the universal approximation theorem still true for other norms like the Sobolev norm $W^{1,1}$? Do these results hold for activation functions other than ReLU? We will answer all of those questions and more with a framework of two expressive powers. The first one is well-known and counts the maximal number of linear regions of a function calculated by a ReLU-network. We will improve the best known bounds for this expressive power. The second one is entirely new.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
On the Approximation Properties of Neural NetworksJonathan W. Siegel, Jinchao Xu · 2019 · Neural Networks (2020) · Résumé IA disponible
Résumé scientifique
We prove two new results concerning the approximation properties of neural networks. Our first result gives conditions under which the outputs of the neurons in a two layer neural network are linearly independent functions. Our second result concerns the rate of approximation of a two layer neural network as the number of neurons increases. We improve upon existing results in the literature by significantly relaxing the required assumptions on the activation function and by providing a better rate of approximation. We also provide a simplified proof that the class of functions represented by a two-layer neural network is dense in any compact set if the activation function is not a polynomial…
-
New Integral representations for the Fox-Wright functions and its applicationsKhaled Mehrez · 2017 · Analyse classique et EDO · Résumé IA disponible
Résumé scientifique
Our aim in this paper is to derive several new integral representations of the Fox-Wright functions. In particular, we give new Laplace and Stieltjes transform for this special functions under a special restriction on parameters. From the positivity conditions for the weight in these representations, we found sufficient conditions to be imposed on the parameters of the Fox-Wright functions that it be completely monotonic. As applications, we derive a class of function related to the Fox H-functions is positive definite and an investigation of a class of the Fox H-function is non-negative. Moreover, we extended the Luke's inequalities and we establish a new Tur\'an type inequalities for the F…
-
Identities for the Ln-transform, the L2n-transform and the P2n transform and their applicationsNese Dernek, Fatih Aylikci · 2014 · Analyse classique et EDO · Résumé IA disponible
Résumé scientifique
In the present paper, the authors introduce several new integral transforms including the Ln-transform, the L2n-transform and P2n-transform generalizations of the classical Laplace transform and the classical Stieltjes transform as respectively. It is shown that the second iterate of the L2n-transform is essentially the P2n-transform. Using this relationship, a few new Parseval-Goldstein type identities are obtained. The theorem and the lemmas that are proven in this article are new useful relations for evaluating infinite integrals of special functions. Some related illustrative examples are also given.
-
Perturbations of Cauchy differencesEszter Gselmann, Tomasz Małolepszy, Janusz Matkowski · 2026 · Analyse classique et EDO · Résumé IA disponible
Résumé scientifique
This paper investigates functional equations arising from perturbations of Cauchy differences. We study equations of the form \[ f(x+y)-f(x)-f(y)=B(x,y) \quad \text{or} \quad f(xy)-f(x)f(y) = B(x,y) \] where $B$ is a biadditive mapping, and also more general cases where the inhomogeneity depends on unknown functions \begin{align*} f(x+y)-f(x)-f(y)&= αx y \\[2.5mm] f(x+y)-f(x)-f(y)&= α(x y)\\[2.5mm] f(x+y)-f(x)-f(y)&= α(x)α(y). \end{align*} Our results extend previous work on the bilinearity of the Cauchy exponential difference by Alzer and Matkowski. We characterize solutions under various structural and regularity assumptions, including additive and exponential Cauchy differences, and show …
-
New elementary formulas for any derivative of any rational functionShahar Nevo, Irina Raichik · 2020 · Analyse classique et EDO · Résumé IA disponible
Résumé scientifique
In this paper we introduce elementary and completely explicit formulas for the derivative of any order of any function of the type 1/p, where p is a polynomial with known zeros.
-
Phase retrieval for solutions of the Schr{ö}dinger equationsPhilippe Jaming · 2025 · SampTA 2025, Jul 2025, Vienne, Austria · Résumé IA disponible
Résumé scientifique
In this note we present several questions about the phase retrieval problem for the Schr{\"o}dinger equation. Some partial answers are given as well as some of the heuristics behind these questions.
-
Multiple basic hypergeometric transformation formulas arising from the balanced duality transformationYasushi Kajihara · 2013 · Analyse classique et EDO · Pas encore résumé
Résumé scientifique
Some multiple hypergeometric transformation formulas arising from the balanced du- ality transformation formula are discussed through the symmetry. Derivations of some transformation formulas with different dimensions are given by taking certain limits of the balanced duality transformation. By combining some of them, some transformation formulas for $A_n$ basic hypergeometric series is given. They include some generalizations of Watson, Sears and ${}_8 W_7$ transformations.
-
Orthogonal Fourier Analysis on DomainsMihail N. Kolountzakis · 2024 · Analyse classique et EDO · Résumé IA disponible
Résumé scientifique
In this paper we go over the history of the Fuglede or Spectral Set Conjecture as it has developed over the last 30 years or so. We do not aim to be exhaustive and we do not cover important areas of development such as the results on the problem in classes of finite groups or the version of the problem that focuses on spectral measures instead of sets. The selection of the material has been strongly influenced by personal taste, history and capabilities. We are trying to be more descriptive than detailed and we point out several open questions.
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Pour l’instant seulement parmi les articles résumés sur ce site : relancez la recherche pour ajouter les travaux antérieurs, les suites et les recommandations de Semantic Scholar.