Tangent Space Separability in Feedforward Neural Networks

12/18/2019
by   Bálint Daróczy, et al.
0

Hierarchical neural networks are exponentially more efficient than their corresponding "shallow" counterpart with the same expressive power, but involve huge number of parameters and require tedious amounts of training. By approximating the tangent subspace, we suggest a sparse representation that enables switching to shallow networks, GradNet after a very early training stage. Our experiments show that the proposed approximation of the metric improves and sometimes even surpasses the achievable performance of the original network significantly even after a few epochs of training the original feedforward network.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset