Model reparametrization for improving variational inference
share
In this article, we propose a strategy to improve variational Bayes inference for a class of models whose variables can be classified as global (common across all observations) or local (observation specific) by using a model reparametrization. In particular, an invertible affine transformation is applied on the local variables so that their posterior dependency on the global variables is minimized. The functional form of this transformation is deduced by approximating the conditional posterior distribution of each local variable given the global variables by a Gaussian distribution via a second order Taylor expansion. Variational inference for the reparametrized model is then obtained using stochastic approximation techniques. Our approach can be readily extended to large datasets via a divide and recombine strategy. Application of the methods is illustrated using generalized linear mixed models.
READ FULL TEXT