research
∙
06/26/2023
Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression
Pretrained transformers exhibit the remarkable ability of in-context lea...
share
research
∙
10/06/2022
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?
Modern deep learning involves training costly, highly overparameterized ...
research
∙
06/02/2022
Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks
A striking observation about iterative magnitude pruning (IMP; Frankle e...
research
∙
07/15/2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training
The recent success of deep learning has partially been driven by trainin...
research
∙
10/28/2020