research
∙
06/01/2023
Exposing Attention Glitches with Flip-Flop Language Modeling
Why do large language models sometimes output factual inaccuracies and e...
share
research
∙
06/01/2023
Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation
Good data augmentation is one of the key factors that lead to the empiri...
research
∙
10/19/2022
Transformers Learn Shortcuts to Automata
Algorithmic reasoning requires capabilities which are most naturally und...
research
∙
02/18/2022
Masked prediction tasks: a parameter identifiability view
The vast majority of work in self-supervised learning, both theoretical ...
research
∙
10/21/2021
Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation
Noise-contrastive estimation (NCE) is a statistically consistent method ...
research
∙
03/03/2021
Contrastive learning of strong-mixing continuous-time stochastic processes
Contrastive learning is a family of self-supervised methods where a mode...
research
∙
06/11/2018