research
∙
09/09/2023
MADLAD-400: A Multilingual And Document-Level Large Audited Dataset
We introduce MADLAD-400, a manually audited, general domain 3T token mon...
share
research
∙
08/17/2023
Reinforced Self-Training (ReST) for Language Modeling
Reinforcement learning from human feedback (RLHF) can improve the qualit...
research
∙
08/14/2023
The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation
Automatic evaluation of machine translation (MT) is a critical tool driv...
research
∙
06/15/2023
Block-State Transformer
State space models (SSMs) have shown impressive results on tasks that re...
research
∙
05/19/2023
Cross-Lingual Supervision improves Large Language Models Pre-training
The recent rapid progress in pre-training Large Language Models has reli...
research
∙
04/18/2023
UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining
Pretrained multilingual large language models have typically used heuris...
research
∙
03/27/2023
Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation
Neural machine translation (NMT) has progressed rapidly over the past se...
research
∙
02/19/2023
Scaling Laws for Multilingual Neural Machine Translation
In this work, we provide a large-scale empirical study of the scaling pr...
research
∙
02/09/2023
Binarized Neural Machine Translation
The rapid scaling of language models is motivating research using low-bi...
research
∙
02/02/2023
The unreasonable effectiveness of few-shot learning for machine translation
We demonstrate the potential of few-shot translation systems, trained wi...
research
∙
01/24/2023
Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interaction
Crosslingual conditional generation (e.g., machine translation) has long...
research
∙
10/01/2022
FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation
We present FRMT, a new dataset and evaluation benchmark for Few-shot Reg...
research
∙
09/23/2022
Do Current Multi-Task Optimization Methods in Deep Learning Even Help?
Recent research has proposed a series of specialized optimization algori...
research
∙
05/09/2022
Building Machine Translation Systems for the Next Thousand Languages
In this paper we share findings from our effort to build practical machi...
research
∙
04/05/2022
PaLM: Scaling Language Modeling with Pathways
Large language models have been shown to achieve remarkable performance ...
research
∙
03/21/2022
XTREME-S: Evaluating Cross-lingual Speech Representations
We introduce XTREME-S, a new benchmark to evaluate universal cross-lingu...
research
∙
03/15/2022
Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation
Multilingual neural machine translation models are trained to maximize t...
research
∙
02/23/2022
Using natural language prompts for machine translation
We explore the use of natural language prompts for controlling various a...
research
∙
02/04/2022
Data Scaling Laws in NMT: The Effect of Noise and Architecture
In this work, we study the effect of varying the architecture and traini...
research
∙
02/01/2022
Examining Scaling and Transfer of Language Model Architectures for Machine Translation
Natural language understanding and generation models follow one of the t...
research
∙
01/09/2022
Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning
Achieving universal translation between all human language pairs is the ...
research
∙
12/13/2021
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has drive...
research
∙
10/08/2021
A Loss Curvature Perspective on Training Instability in Deep Learning
In this work, we study the evolution of the loss Hessian across many cla...
research
∙
09/21/2021
Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
Document-level neural machine translation (DocNMT) delivers coherent tra...
research
∙
09/19/2021
Towards Zero-Label Language Learning
This paper explores zero-label learning in Natural Language Processing (...
research
∙
09/16/2021
Scaling Laws for Neural Machine Translation
We present an empirical study of scaling properties of encoder-decoder T...
research
∙
09/13/2021
Evaluating Multiway Multilingual NMT in the Turkic Languages
Despite the increasing number of large and comprehensive machine transla...
research
∙
09/09/2021
A Large-Scale Study of Machine Translation in the Turkic Languages
Recent advances in neural machine translation (NMT) have pushed the qual...
research
∙
07/30/2021
Towards Universality in Multilingual Text Rewriting
In this work, we take the first steps towards building a universal rewri...
research
∙
04/15/2021
XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation
Machine learning has brought striking advances in multilingual natural l...
research
∙
03/22/2021
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
With the success of large-scale pre-training and multilingual modeling i...
research
∙
03/11/2021
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution
We propose a straightforward vocabulary adaptation scheme to extend the ...
research
∙
02/26/2021
Gradient-guided Loss Masking for Neural Machine Translation
To mitigate the negative effect of low quality training data on the perf...
research
∙
10/23/2020
Rapid Domain Adaptation for Machine Translation with Monolingual Data
One challenge of machine translation is how to quickly adapt to unseen d...
research
∙
10/20/2020
Towards End-to-End In-Image Neural Machine Translation
In this paper, we offer a preliminary investigation into the task of in-...
research
∙
10/20/2020
Complete Multilingual Neural Machine Translation
Multilingual Neural Machine Translation (MNMT) models are commonly train...
research
∙
10/15/2020
Explicit Alignment Objectives for Multilingual Bidirectional Encoders
Pre-trained cross-lingual encoders such as mBERT (Devlin et al., 2019) a...
research
∙
10/12/2020
Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models
Massively multilingual models subsuming tens or even hundreds of languag...
research
∙
09/23/2020
Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages
Unsupervised translation has reached impressive performance on resource-...
research
∙
06/30/2020
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
Neural network scaling has been critical for improving the model quality...
research
∙
05/11/2020
Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation
Over the last few years two promising research directions in low-resourc...
research
∙
03/24/2020
XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization
Much recent progress in applications of machine learning models to NLP h...
research
∙
02/17/2020
On the Discrepancy between Density Estimation and Sequence Generation
Many sequence-to-sequence generation tasks, including machine translatio...
research
∙
02/17/2020
Controlling Computation versus Quality for Neural Sequence Models
Most neural networks utilize the same amount of compute for every exampl...
research
∙
10/30/2019
Fill in the Blanks: Imputing Missing Sentences for Larger-Context Neural Machine Translation
Most neural machine translation systems still translate sentences in iso...
research
∙
10/15/2019
On the Importance of Word Boundaries in Character-level Neural Machine Translation
Neural Machine Translation (NMT) models generally perform translation us...
research
∙
09/18/2019
Simple, Scalable Adaptation for Neural Machine Translation
Fine-tuning pre-trained Neural Machine Translation (NMT) models is the d...
research
∙
09/13/2019
Adaptive Scheduling for Multi-Task Learning
To train neural machine translation models simultaneously on multiple ta...
research
∙
09/05/2019
Investigating Multilingual NMT Representations at Scale
Multilingual Neural Machine Translation (NMT) models have yielded large ...
research
∙
09/01/2019