research
∙
08/15/2023
RAVEN: In-Context Learning with Retrieval Augmented Encoder-Decoder Language Models
In this paper, we investigate the in-context learning ability of retriev...
share
research
∙
04/13/2023
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Large decoder-only language models (LMs) can be largely improved in term...
research
∙
02/14/2023
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models
Pretrained large language models have become indispensable for solving v...
research
∙
02/09/2023
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
Augmenting pretrained language models (LMs) with a vision encoder (e.g.,...
research
∙
10/25/2022
Evaluating Parameter Efficient Learning for Generation
Parameter efficient learning methods (PERMs) have recently gained signif...
research
∙
10/12/2022
Context Generation Improves Open Domain Question Answering
Closed-book question answering (QA) requires a model to directly answer ...
research
∙
10/06/2022
Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models
We explore the idea of compressing the prompts used to condition languag...
research
∙
09/12/2022
FP8 Formats for Deep Learning
FP8 is a natural progression for accelerating deep learning training inf...
research
∙
06/09/2022
Factuality Enhanced Language Models for Open-Ended Text Generation
Pretrained language models (LMs) are susceptible to generate text with n...
research
∙
05/10/2022
Reducing Activation Recomputation in Large Transformer Models
Training large transformer models is one of the most important computati...
research
∙
03/16/2022
Multi-Stage Prompting for Knowledgeable Dialogue Generation
Existing knowledge-grounded dialogue systems typically use finetuned ver...
research
∙
02/08/2022
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
Pre-trained language models (LMs) are shown to easily generate toxic lan...
research
∙
01/28/2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Pretrained general-purpose language models can achieve state-of-the-art ...
research
∙
12/15/2021
Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases
Detecting social bias in text is challenging due to nuance, subjectivity...
research
∙
07/05/2021
Long-Short Transformer: Efficient Transformers for Language and Vision
Transformers have achieved success in both language and vision domains. ...
research
∙
04/09/2021
Efficient Large-Scale Language Model Training on GPU Clusters
Large language models have led to state-of-the-art accuracies across a r...
research
∙
01/02/2021
End-to-End Training of Neural Retrievers for Open-Domain Question Answering
Recent work on training neural retrievers for open-domain question answe...
research
∙
10/20/2020
Local Knowledge Powered Conversational Agents
State-of-the-art conversational agents have advanced significantly in co...
research
∙
10/12/2020
BioMegatron: Larger Biomedical Domain Language Model
There has been an influx of biomedical domain-specific language models, ...
research
∙
10/02/2020
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models
Existing pre-trained large language models have shown unparalleled gener...
research
∙
05/13/2020
Large Scale Multi-Actor Generative Dialog Modeling
Non-goal oriented dialog agents (i.e. chatbots) aim to produce varying a...
research
∙
03/02/2020
Style Example-Guided Text Generation using Generative Adversarial Transformers
We introduce a language generative model framework for generating a styl...
research
∙
02/22/2020
Training Question Answering Models From Synthetic Data
Question and answer generation is a data augmentation method that aims t...
research
∙
12/25/2019
Neural ODEs for Image Segmentation with Level Sets
We propose a novel approach for image segmentation that combines Neural ...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using GPU Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
06/13/2019
Unsupervised Video Interpolation Using Cycle Consistency
Learning to synthesize high frame rate videos via interpolation requires...
research
∙
10/25/2017
Trace norm regularization and faster inference for embedded speech recognition RNNs
We propose and evaluate new techniques for compressing and speeding up d...
research
∙
02/25/2017