research
∙
09/13/2023
Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs
Most interpretability research in NLP focuses on understanding the behav...
share
research
∙
05/24/2023
Dynamic Masking Rate Schedules for MLM Pretraining
Most works on transformers trained with the Masked Language Modeling (ML...
research
∙
11/01/2022
Reduce, Reuse, Recycle: Improving Training Efficiency with Distillation
Methods for improving the efficiency of deep network training (i.e. the ...
research
∙
10/22/2020
Towards falsifiable interpretability research
Methods for understanding the decisions of and mechanisms underlying dee...
research
∙
10/14/2020
Linking average- and worst-case perturbation robustness via class selectivity and dimensionality
Representational sparsity is known to affect robustness to input perturb...
research
∙
07/08/2020
On the relationship between class selectivity, dimensionality, and robustness
While the relative trade-offs between sparse and distributed representat...
research
∙
03/03/2020