Universal approximation properties of shallow quadratic neural networks

10/04/2021
by   Leon Frischauf, et al.
0

In this paper we propose a new class of neural network functions which are linear combinations of compositions of activation functions with quadratic functions, replacing standard affine linear functions, often called neurons. We show the universality of this approximation and prove convergence rates results based on the theory of wavelets and statistical learning. We investigate the efficiency of our new approach numerically for simple test cases, by showing that it requires less numbers of neurons that standard affine linear neural networks. Similar observations are made when comparing deep (multi-layer) networks.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset