Convergence Analysis of Function Approximation: From Classical Polynomials to Neural Networks
DOI:
https://doi.org/10.24843/JMAT.2026.v16.i01.p199Keywords:
Weierstrass approximation theorem, Bernstein polynomials, Fourier series, Approximating, Stability analysisAbstract
Function approximation lies at the boundary between classical analysis and modern computing, although the literature tends to consider its classical and modern branches as independent. In this paper, we propose that polynomial neural networks with a single hidden layer, methods, and Hilbert space projections are not only identical but also share a fundamental structure that can be explicitly expressed. We revisit Weiserstrasse and Bernstein's probabilistic construction theories, develop a Hilbert space projection framework, and investigate numerical problems such as node positioning and least-squares stability. All these lines converge in a discussion of Sybenko's theory of total approximation, which can be read as a classical result rather than a break from it. Numerical experiments on the Fourier square wave reconstruction and the Fourier square wave reconstruction are accompanied by theoretical development on . There is an error and stability at which the paper concludes. The analysis can be applied to all three approximation models.
References
[1] R. Siegmund-Schultze, “Weierstraß’s Approximation Theorem (1885) and his 1886 lecture course revisited,” in Karl Weierstraß (1815–1897), Wiesbaden: Springer Fachmedien Wiesbaden, 2016, pp. 219–268. https://doi.org/10.1007/978-3-658-10619-5_8
[2] R. S. Rajawat, K. K. Singh, and V. N. Mishra, “Approximation by modified Bernstein polynomials based on real parameters,” Math. Found. Comput., vol. 7, no. 3, pp. 297–309, 2024. Doi: 10.3934/mfc.2023005
[3] F. Özger, H. M. Srivastava, and S. A. Mohiuddine, “Approximation of functions by a new class of generalized Bernstein–Schurer operators,” Rev. R. Acad. Cienc. Exactas Fis. Nat. Ser. A Mat. RACSAM, vol. 114, no. 4, 2020. https://doi.org/10.1007/s13398-020-00903-6
[4] Q. B. Cai and R. Aslan, “On a new construction of generalized q-Bernstein polynomials based on shape parameter λ. Symmetry,” vol. 13, 2021. https://doi.org/10.3390/sym13101919
[5] R. Aslan and M. Mursaleen, “Some approximation results on a class of new type λ-Bernstein polynomials,” J. Math. Inequal, vol. 16, no. 2, pp. 445–462, 2022. doi:10.7153/jmi-2022-16-32
[6] R. A. Kennedy and P. Sadeghi, Hilbert space methods in signal processing. Cambridge University Press, 2013.
[7] J. Muscat, “Hilbert Spaces,” in Functional Analysis: An Introduction to Metric Spaces, Hilbert Spaces, and Banach Algebras, Cham: Springer International Publishing, 2024, pp. 197–247. https://doi.org/10.1007/978-3-031-27537-1
[8] L. Debnath and P. Mikusinski, Introduction to Hilbert spaces with applications, 3rd ed. San Diego, CA: Academic Press, 2005.
[9] E. Provenzi, From Euclidean to Hilbert Spaces: Introduction to Functional Analysis and Its Applications. John Wiley & Sons, 2021.
[10] G. Cybenko, “Approximation by superpositions of a sigmoidal function,” Mathematics of control, signals and systems, vol. 2, pp. 303–314, 1989. https://doi.org/10.1007/BF02551274
[11] K. Hornik, M. Stinchcombe, and H. White, “Multilayer feedforward networks are universal approximators,” Neural networks, vol. 2, no. 5, pp. 359–366, 1989. https://doi.org/10.1016/0893-6080(89)90020-8
[12] E. Paluzo-Hidalgo, R. Gonzalez-Diaz, and M. A. Gutiérrez-Naranjo, “Two-hidden-layer feed-forward networks are universal approximators: A constructive approach,” Neural Netw., vol. 131, pp. 29–36, 2020. https://doi.org/10.1016/j.neunet.2020.07.021
[13] Z. Yan, B. Chen, Y. Liu, and Q. Ye, “Expandable residual approximation for knowledge distillation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 37, no. 1, pp. 191–203, 2026.
[14] Z. Fan et al., “Bernstein bounds for caustics,” ACM Trans. Graph., vol. 44, no. 4, pp. 1–15, 2025. DOI: 10.1109/TNNLS.2025.3602118
[15] T. Akhobadze and S. Zviadadze, “Approximation properties of trigonometric Fourier series in generalized variation classes,” Advances in Operator Theory, vol. 10, 2025. https://doi.org/10.1007/s43036-024-00392-z
[16] T. Alazard, “Fourier Series,” in Analysis and Partial Differential Equations, Cham: Springer Nature Switzerland, 2024, pp. 95–112. https://doi.org/10.1007/978-3-031-70909-8
[17] S. Tang, B. Li, and H. Yu, “ChebNet: Efficient and stable constructions of deep neural networks with rectified power units via Chebyshev approximation,” Commun. Math. Stat., 2024. https://doi.org/10.1007/s40304-023-00392-0
[18] C. F. Van Loan and J. P. Vokt, “Approximating matrices with multiple symmetries,” SIAM J. Matrix Anal. Appl., vol. 36, no. 3, pp. 974–993, 2015. https://doi.org/10.1137/140986347
[19] Å. Björck, Numerical methods for least squares problems, 2nd ed. New York, NY: Society for Industrial & Applied Mathematics, 2024.
[20] M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken, “Multilayer feedforward networks with a nonpolynomial activation function can approximate any function,” Neural Netw., vol. 6, no. 6, pp. 861–867, 1993. https://doi.org/10.1016/S0893-6080(05)80131-5
[21] A. Amini-Harandi, M. Fakhar, and H. R. Hajisharifi, “Some generalizations of the Weierstrass theorem,” SIAM J. Optim., vol. 26, no. 4, pp. 2847–2862, 2016. https://doi.org/10.1016/S0893-6080(05)80131-5
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Ekhlas Annon Mousa
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
