Sparse Latent Factor Models with Pure Variables for Overlapping Clustering
share
The problem of overlapping variable clustering, ubiquitous in data science, is that of finding overlapping sub-groups of a p-dimensional random vector X, from a sample of size n of observations on X. Typical solutions are algorithmic in nature, and little is known about the statistical guarantees of the estimated clusters, as most algorithms are not model-based. This work introduces a novel method, LOVE, based on a sparse Latent factor model, with correlated factors, and with pure variables, for OVErlapping clustering with statistical guarantees. The model is used to define the population level clusters as groups of those components of X that are associated, via a sparse allocation matrix, with the same unobservable latent factor, and multi-factor association is allowed. Clusters are respectively anchored by components of X, called pure variables, that are associated with only one latent factor. We prove that the existence of pure variables is a sufficient, and almost necessary, assumption for the identifiability of the allocation matrix, in sparse latent factor models. Consequently, model-based clusters can be uniquely defined, and provide a bona fide estimation target. LOVE estimates first the set of pure variables, and the number of clusters, via a novel method that has low computational complexity of order p2. Each cluster, anchored by pure variables, is then further populated with components of X according to the sparse estimates of the allocation matrix. The latter are obtained via a new, computationally efficient, estimation method tailored to the structure of this problem. The combined procedure yields rate-optimal estimates of the allocation matrix and consistent estimators of the number of clusters.
READ FULL TEXT