Unsupervised Ensemble Selection for Multilayer Bootstrap Networks
share
Multilayer bootstrap network (MBN), which is a recent simple unsupervised deep model, is sensitive to its network structure. How to select a proper network structure that may be dramatically different in different applications is a hard issue, given little prior knowledge of data. In this paper, we explore ensemble learning and selection techniques for determining the optimal network structure of MBN automatically. Specifically, we first propose an MBN ensemble (MBN-E) algorithm which concatenates the sparse outputs of a set of MBN base models with different network structures into a new representation. Then, we take the new representation as a reference for selecting the optimal MBN base models. The ensemble selection criteria can be categorized into two classes. The first kind employs optimization-like selection criteria, under the assumption that the number of classes of data is known as a prior. The second kind proposes distribution divergence criteria, when such a prior is unavailable. Experimental results on several benchmark datasets show that MBN-E yields good performance that is close to the optimal performance of MBN, while the ensemble selection techniques for MBN-E can further improve the performance. More importantly, MBN-E and its ensemble selection techniques maintain the simple formulation of MBN, and act like off-the-shelf methods that reach the state-of-the-art performance without manual hyperparameter tuning. The source code is available at http://www.xiaolei-zhang.net/mbn-e.htm.
READ FULL TEXT