Equivalence of approximation by convolutional neural networks and fully-connected networks
share
Convolutional neural networks are the most widely used type of neural networks in applications. In mathematical analysis, however, mostly fully-connected networks are studied. In this paper, we establish a connection between both network architectures. Using this connection, we show that all upper and lower bounds concerning approximation rates of fully-connected neural networks for functions f ∈C---for an arbitrary function class C---translate to essentially the same bounds on approximation rates of convolutional neural networks for functions f ∈C^equi, with the class C^equi consisting of all translation equivariant functions whose first coordinate belongs to C.
READ FULL TEXT