Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem
share
We propose a new algorithm for sparse estimation of eigenvectors in generalized eigenvalue problems (GEP). The GEP arises in a number of standard and modern statistical methods, including principal component analysis (PCA) and multiclass linear discriminant analysis (multiclass LDA). We propose to modify the standard generalized orthogonal iteration with a sparsity-inducing penalty for the eigenvectors. To achieve this goal, we generalize the equation-solving step of orthogonal iteration to a penalized convex optimization problem. The resulting algorithm, called penalized orthogonal iteration, provides fast computation and accurate estimation of the true eigenspace, when it is sparse. Numerical studies reveal that the proposed algorithms are competitive, and that our tuning procedure works well. We focus on the data situations of PCA and multiclass LDA in simulation studies. A real data example from a genetic study further shows the advantage of the sparse eigenvector estimation.
READ FULL TEXT