research
∙
05/23/2023
Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model
Large and sparse feed-forward networks (S-FFN) such as Mixture-of-Expert...
share
research
∙
05/03/2022
Improving In-Context Few-Shot Learning via Self-Supervised Training
Self-supervised pretraining has made few-shot learning possible for many...
research
∙
04/03/2022
PERFECT: Prompt-free and Efficient Few-shot Learning with Language Models
Current methods for few-shot fine-tuning of pretrained masked language m...
research
∙
12/20/2021
Few-shot Learning with Multilingual Language Models
Large-scale autoregressive language models such as GPT-3 are few-shot le...
research
∙
11/26/2021
Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs
Do language models have beliefs about the world? Dennett (1995) famously...
research
∙
01/01/2021
Multi-task Retrieval for Knowledge-Intensive Tasks
Retrieving relevant contexts from a large corpus is a crucial step for t...
research
∙
04/29/2020
General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
The state of the art on many NLP tasks is currently achieved by large pr...
research
∙
12/20/2019
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
Recent breakthroughs of pretrained language models have shown the effect...
research
∙
12/14/2019
SemEval-2013 Task 2: Sentiment Analysis in Twitter
In recent years, sentiment analysis in social media has attracted a lot ...
research
∙
12/06/2019
SemEval-2014 Task 9: Sentiment Analysis in Twitter
We describe the Sentiment Analysis in Twitter task, ran as part of SemEv...
research
∙
12/05/2019
SemEval-2015 Task 10: Sentiment Analysis in Twitter
In this paper, we describe the 2015 iteration of the SemEval shared task...
research
∙
12/03/2019
SemEval-2016 Task 4: Sentiment Analysis in Twitter
This paper discusses the fourth year of the “Sentiment Analysis in Twitt...
research
∙
11/05/2019
Unsupervised Cross-lingual Representation Learning at Scale
This paper shows that pretraining multilingual language models at scale ...
research
∙
11/04/2019
Emerging Cross-lingual Structure in Pretrained Language Models
We study the problem of multilingual masked language modeling, i.e. the ...
research
∙
09/16/2019
Bridging the domain gap in cross-lingual document classification
The scarcity of labeled training data often prohibits the internationali...
research
∙
07/26/2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Language model pretraining has led to significant performance gains but ...
research
∙
04/09/2019
Knowledge-Augmented Language Model and its Application to Unsupervised Named-Entity Recognition
Traditional language models are unable to efficiently model entity names...
research
∙
09/13/2018
XNLI: Evaluating Cross-lingual Sentence Representations
State-of-the-art natural language processing systems rely on supervision...
research
∙
09/01/2018