Heaviside function as an activation function
-
Naoya Hatano
Abstract
The uniform closure of the neural networks with Heaviside activation is specified.
Funding source: Japan Society for the Promotion of Science
Award Identifier / Grant number: 19K14581
Award Identifier / Grant number: 19K03546
Funding statement: This work was supported by a JST CREST Grant (Number JPMJCR1913, Japan). This work was also supported by the RIKEN Junior Research Associate Program. The second author is supported by a Grant-in-Aid for Young Scientists Research (No. 19K14581), Japan Society for the Promotion of Science. The fourth author is supported by a Grant-in-Aid for Scientific Research (C) (No. 19K03546), Japan Society for the Promotion of Science and People’s Friendship University of Russia.
Acknowledgements
The authors are grateful to the anonymous referee for his careful reading of this paper.
References
[1] G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signals Systems 2 (1989), no. 4, 303–314. 10.1007/BF02551274Search in Google Scholar
[2] B. Hanin, Universal function approximation by deep neural nets with bounded width and ReLU activations, Mathematics 7 (2019), no. 10, Paper No. 992. 10.3390/math7100992Search in Google Scholar
[3] N. Hatano, M. Ikeda, M. Ishikawa and Y. Sawano, A global universality of two-layer neural networks with ReLU activations, J. Funct. Spaces 2021 (2021), Article ID 6637220. 10.1155/2021/6637220Search in Google Scholar
[4] P. Kainen, V. Kůrková and A. Vogt, Best approximation by Heaviside perceptron networks, Neural Networks 13 (2000), no. 7, 695–697. 10.1016/S0893-6080(00)00056-3Search in Google Scholar
[5] A. Pinkus, Approximation theory of the MLP model in neural networks, Acta Numer. 8 (1999), 143–195. 10.1017/S0962492900002919Search in Google Scholar
[6] S. Sonoda and N. Murata, Neural network with unbounded activation functions is universal approximator, Appl. Comput. Harmon. Anal. 43 (2017), no. 2, 233–268. 10.1016/j.acha.2015.12.005Search in Google Scholar
© 2022 Walter de Gruyter GmbH, Berlin/Boston
Articles in the same Issue
- Frontmatter
- Heaviside function as an activation function
- Krylov solvability under perturbations of abstract inverse linear problems
- Existence of solutions for a three-point Hadamard fractional resonant boundary value problem
- Positive solutions to mixed fractional p-Laplacian boundary value problems
- Maximum number of limit cycles for generalized Kukles differential system
- Stabilization of a 1-D transmission problem for the Rayleigh beam and string with localized frictional damping
- Certain multiplier results on Bp spaces
- Remarks on the Belitskii–Lyubich and discrete Markus–Yamabe conjectures
- On deferred statistical convergence of complex uncertain sequences
- Local existence and uniqueness of regular solutions to a Landau–Lifshitz–Bloch equation with applied current
- Converse Ohlin’s lemma for convex and strongly convex functions
- Approximate controllability results in α-norm for some partial functional integrodifferential equations with nonlocal initial conditions in Banach spaces
- Direct proofs of intrinsic properties of prox-regular sets in Hilbert spaces
- On the continuity properties of the Lp balls
- On L 𝔮 convergence of the Hamiltonian Monte Carlo
- Radii of starlikeness and convexity of generalized k-Bessel functions
- An iterative technique for solving split equality monotone variational inclusion and fixed point problems
Articles in the same Issue
- Frontmatter
- Heaviside function as an activation function
- Krylov solvability under perturbations of abstract inverse linear problems
- Existence of solutions for a three-point Hadamard fractional resonant boundary value problem
- Positive solutions to mixed fractional p-Laplacian boundary value problems
- Maximum number of limit cycles for generalized Kukles differential system
- Stabilization of a 1-D transmission problem for the Rayleigh beam and string with localized frictional damping
- Certain multiplier results on Bp spaces
- Remarks on the Belitskii–Lyubich and discrete Markus–Yamabe conjectures
- On deferred statistical convergence of complex uncertain sequences
- Local existence and uniqueness of regular solutions to a Landau–Lifshitz–Bloch equation with applied current
- Converse Ohlin’s lemma for convex and strongly convex functions
- Approximate controllability results in α-norm for some partial functional integrodifferential equations with nonlocal initial conditions in Banach spaces
- Direct proofs of intrinsic properties of prox-regular sets in Hilbert spaces
- On the continuity properties of the Lp balls
- On L 𝔮 convergence of the Hamiltonian Monte Carlo
- Radii of starlikeness and convexity of generalized k-Bessel functions
- An iterative technique for solving split equality monotone variational inclusion and fixed point problems
