A Common Framework for Natural Gradient and Taylor based Optimisation using Manifold Theory

03/26/2018

∙

by   Adnan Haider, et al.

∙

0

∙

share

This technical report constructs a theoretical framework to relate standard Taylor approximation based optimisation methods with Natural Gradient (NG), a method which is Fisher efficient with probabilistic models. Such a framework will be shown to also provide mathematical justification to combine higher order methods with the method of NG.