research
∙
03/20/2023
Unit Scaling: Out-of-the-Box Low-Precision Training
We present unit scaling, a paradigm for designing deep learning models t...
share
research
∙
11/22/2022
BESS: Balanced Entity Sampling and Sharing for Large-Scale Knowledge Graph Completion
We present the award-winning submission to the WikiKG90Mv2 track of OGB-...
research
∙
06/06/2022
8-bit Numerical Formats for Deep Neural Networks
Given the current trend of increasing size and complexity of machine lea...
research
∙
08/13/2021
Towards Structured Dynamic Sparse Pre-Training of BERT
Identifying algorithms for computational efficient unsupervised training...
research
∙
06/10/2021
GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
Attention based language models have become a critical component in stat...
research
∙
06/07/2021
Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence
We investigate the reasons for the performance degradation incurred with...
research
∙
06/07/2021
Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training
Much recent research has been dedicated to improving the efficiency of t...
research
∙
12/07/2020
Parallel Training of Deep Networks with Local Updates
Deep learning models trained on large data sets have been widely success...
research
∙
11/09/2020
Improving Neural Network Training in Low Dimensional Random Bases
Stochastic Gradient Descent (SGD) has proven to be remarkably effective ...
research
∙
04/20/2018