Efficient Representation for Natural Language Processing via Kernelized Hashcodes
share
Kernel similarity functions have been successfully applied in classification models such as Support Vector Machines, Gaussian Processes and k-Nearest Neighbors (kNN), but found to be computationally expensive for Natural Language Processing (NLP) tasks due to the cost of computing kernel similarities between discrete natural language structures. A well-known technique, Kernelized Locality Sensitive Hashing (KLSH), allows for an approximate computation of kNN graphs and significantly reduces the number of kernel computations; however, applying KLSH to other classifiers have not been explored. In this paper, we propose to use random subspaces of KLSH codes for constructing an efficient representation that preserves fine-grained structure of the data and is suitable for general classification methods. Further, we proposed an approach for optimizing KLSH model for supervised classification problems, by maximizing a variational lower bound on the mutual information between the KLSH codes (feature vectors) and the class labels.We apply the proposed approach to the task of extracting information about bio-molecular interactions from the semantic parsing of scientific papers. Our empirical results on a variety of datasets demonstrate significant improvements over the state of the art.
READ FULL TEXT