research
∙
01/28/2023
Supervision Complexity and its Role in Knowledge Distillation
Despite the popularity and efficacy of knowledge distillation, there is ...
share
research
∙
01/27/2023
EmbedDistill: A Geometric Knowledge Distillation for Information Retrieval
Large neural models (such as Transformers) achieve state-of-the-art perf...
research
∙
05/19/2021
Balancing Robustness and Sensitivity using Feature Contrastive Learning
It is generally believed that robust training of extremely large network...
research
∙
02/05/2021
On the Reproducibility of Neural Network Predictions
Standard training techniques for neural networks involve multiple source...
research
∙
05/31/2020
Evaluations and Methods for Explanation through Robustness Analysis
Among multiple ways of interpreting a machine learning model, measuring ...
research
∙
05/21/2020
Why distillation helps: a statistical perspective
Knowledge distillation is a technique for improving the performance of a...
research
∙
12/06/2019
Why ADAM Beats SGD for Attention Models
While stochastic gradient descent (SGD) is still the de facto algorithm ...
research
∙
01/15/2013
Matrix Approximation under Local Low-Rank Assumption
Matrix approximation is a common tool in machine learning for building a...
research
∙
03/06/2010