research
∙
10/07/2022
Understanding Transformer Memorization Recall Through Idioms
To produce accurate predictions, language models (LMs) must balance betw...
share
research
∙
03/30/2022
Transformer Language Models without Positional Encodings Still Learn Positional Information
Transformers typically require some form of positional encoding, such as...
research
∙
01/10/2022
SCROLLS: Standardized CompaRison Over Long Language Sequences
NLP benchmarks have largely focused on short texts, such as sentences an...
research
∙
04/19/2021
Can Latent Alignments Improve Autoregressive Machine Translation?
Latent alignment objectives such as CTC and AXE significantly improve no...
research
∙
03/09/2021