research
∙
04/25/2023
Pretrain on just structure: Understanding linguistic inductive biases using transfer learning
Both humans and transformer language models are able to learn language w...
share
research
∙
10/11/2022
Multilingual BERT has an accent: Evaluating English influences on fluency in multilingual models
While multilingual language models can improve NLP performance on low-re...
research
∙
03/11/2022
When classifying grammatical role, BERT doesn't care about word order... except when it matters
Because meaning can often be inferred from lexical semantics alone, word...
research
∙
02/24/2022
Oolong: Investigating What Makes Crosslingual Transfer Hard with Controlled Studies
Little is known about what makes cross-lingual transfer hard, since fact...
research
∙
03/22/2021
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
With the success of large-scale pre-training and multilingual modeling i...
research
∙
01/26/2021
Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERT
We investigate how Multilingual BERT (mBERT) encodes grammar by examinin...
research
∙
04/30/2020