research
∙
04/18/2023
UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining
Pretrained multilingual large language models have typically used heuris...
share
research
∙
12/20/2022
Character-Aware Models Improve Visual Text Rendering
Current image generation models struggle to reliably produce well-formed...
research
∙
10/01/2022
FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation
We present FRMT, a new dataset and evaluation benchmark for Few-shot Reg...
research
∙
09/29/2022
Bidirectional Language Models Are Also Few-shot Learners
Large language models such as GPT-3 (Brown et al., 2020) can perform arb...
research
∙
08/10/2022
Reducing Retraining by Recycling Parameter-Efficient Prompts
Parameter-efficient methods are able to use a single frozen pre-trained ...
research
∙
05/25/2022
Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation
In this paper, we explore the challenging problem of performing a genera...
research
∙
10/15/2021
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
As pre-trained language models have gotten larger, there has been growin...
research
∙
08/19/2021
Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models
We provide the first exploration of text-to-text transformers (T5) sente...
research
∙
07/30/2021
Towards Universality in Multilingual Text Rewriting
In this work, we take the first steps towards building a universal rewri...
research
∙
06/03/2021
nmT5 – Is parallel data still relevant for pre-training massively multilingual language models?
Recently, mT5 - a massively multilingual version of T5 - leveraged a uni...
research
∙
05/28/2021
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Most widely-used pre-trained language models operate on sequences of tok...
research
∙
04/18/2021
The Power of Scale for Parameter-Efficient Prompt Tuning
In this work, we explore "prompt tuning", a simple yet effective mechani...
research
∙
04/15/2021
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation
Machine learning has brought striking advances in multilingual natural l...
research
∙
03/11/2021
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution
We propose a straightforward vocabulary adaptation scheme to extend the ...
research
∙
10/22/2020
Multilingual Synthetic Question and Answer Generation for Cross-Lingual Reading Comprehension
We propose a simple method to generate large amounts of multilingual que...
research
∙
10/22/2020
mT5: A massively multilingual pre-trained text-to-text transformer
The recent "Text-to-Text Transfer Transformer" (T5) leveraged a unified ...
research
∙
10/08/2020
TextSETTR: Label-Free Text Style Extraction and Tunable Targeted Restyling
We present a novel approach to the problem of text style transfer. Unlik...
research
∙
05/05/2020
MultiReQA: A Cross-Domain Evaluation for Retrieval Question Answering Models
Retrieval question answering (ReQA) is the task of retrieving a sentence...
research
∙
04/11/2020
LAReQA: Language-agnostic answer retrieval from a multilingual pool
We present LAReQA, a challenging new benchmark for language-agnostic ans...
research
∙
08/27/2019
Bridging the Gap for Tokenizer-Free Language Models
Purely character-based language models (LMs) have been lagging in qualit...
research
∙
07/10/2019
ReQA: An Evaluation for End-to-End Answer Retrieval Models
Popular QA benchmarks like SQuAD have driven progress on the task of ide...
research
∙
07/09/2019
Multilingual Universal Sentence Encoder for Semantic Retrieval
We introduce two pre-trained retrieval focused multilingual sentence enc...
research
∙
08/09/2018
Character-Level Language Modeling with Deeper Self-Attention
LSTMs and other RNN variants have shown strong performance on character-...
research
∙
04/20/2018
Learning Semantic Textual Similarity from Conversations
We present a novel approach to learn representations for sentence-level ...
research
∙
03/29/2018