research
∙
05/24/2023
Flan-MoE: Scaling Instruction-Finetuned Language Models with Sparse Mixture of Experts
The explosive growth of language models and their applications have led ...
share
research
∙
05/22/2023
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, Toxicity
Pretraining is the preliminary and fundamental step in developing capabl...
research
∙
03/07/2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities
Foundation models pretrained on diverse data at scale have demonstrated ...
research
∙
03/07/2023
Larger language models do in-context learning differently
We study how in-context learning (ICL) in language models is affected by...
research
∙
01/31/2023
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
We study the design decisions of publicly available instruction tuning m...
research
∙
12/26/2022
Large Language Models Encode Clinical Knowledge
Large language models (LLMs) have demonstrated impressive capabilities i...
research
∙
11/03/2022
Inverse scaling can become U-shaped
Although scaling language models improves performance on a range of task...
research
∙
10/20/2022
Transcending Scaling Laws with 0.1
Scaling language models improves performance but comes with significant ...
research
∙
10/11/2022
Mind's Eye: Grounded Language Model Reasoning through Simulation
Successful and effective communication between humans and AI relies on a...
research
∙
10/06/2022
Language Models are Multilingual Chain-of-Thought Reasoners
We evaluate the reasoning abilities of large language models in multilin...
research
∙
07/02/2022
Rationale-Augmented Ensembles in Language Models
Recent research has shown that rationales, or step-by-step chains of tho...
research
∙
05/21/2022
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
We propose a novel prompting strategy, least-to-most prompting, that ena...
research
∙
04/05/2022
PaLM: Scaling Language Modeling with Pathways
Large language models have been shown to achieve remarkable performance ...
research
∙
03/21/2022
Self-Consistency Improves Chain of Thought Reasoning in Language Models
We explore a simple ensemble strategy, self-consistency, that significan...
research
∙
01/28/2022
Chain of Thought Prompting Elicits Reasoning in Large Language Models
Although scaling up language model size has reliably improved performanc...
research
∙
01/28/2022
Calibrating Histopathology Image Classifiers using Label Smoothing
The classification of histopathology images fundamentally differs from t...
research
∙
09/14/2021
Frequency Effects on Syntactic Rule Learning in Transformers
Pre-trained language models perform well on a variety of linguistic task...
research
∙
09/12/2021
Good-Enough Example Extrapolation
This paper asks whether extrapolating the hidden space distribution of t...
research
∙
09/08/2021
A Recipe For Arbitrary Text Style Transfer with Large Language Models
In this paper, we leverage large language models (LMs) to perform zero-s...
research
∙
09/03/2021
Finetuned Language Models Are Zero-Shot Learners
This paper explores a simple method for improving the zero-shot learning...
research
∙
08/19/2021
Language Model Augmented Relevance Score
Although automated metrics are commonly used to evaluate NLG systems, th...
research
∙
08/17/2021
Modulating Language Models with Emotions
Generating context-aware language that embodies diverse emotions is an i...
research
∙
06/30/2021
The MultiBERTs: BERT Reproductions for Robustness Analysis
Experiments with pretrained models such as BERT are often based on a sin...
research
∙
05/15/2021
A Cognitive Regularizer for Language Modeling
The uniform information density (UID) hypothesis, which posits that spea...
research
∙
04/30/2021
Mitigating Political Bias in Language Models Through Reinforced Calibration
Current large-scale language models can be politically biased as a resul...
research
∙
03/12/2021
Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning
Few-shot text classification is a fundamental NLP task in which a model ...
research
∙
01/29/2021
A Petri Dish for Histopathology Image Analysis
With the rise of deep learning, there has been increased interest in usi...
research
∙
01/14/2021
Text Augmentation in a Multi-Task View
Traditional data augmentation aims to increase the coverage of the input...
research
∙
12/07/2020
An Empirical Survey of Unsupervised Text Representation Methods on Twitter Data
The field of NLP has seen unprecedented achievements in recent years. Mo...
research
∙
09/29/2020
Learn like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification
Applying curriculum learning requires both a range of difficulty in data...
research
∙
05/26/2020
What Are People Asking About COVID-19? A Question Classification Dataset
We present COVID-Q, a set of 1,690 questions about COVID-19 from 13 sour...
research
∙
04/27/2020
Difficulty Translation in Histopathology Images
The unique nature of histopathology images opens the door to domain-spec...
research
∙
10/13/2019
Generative Image Translation for Data Augmentation in Colorectal Histopathology Images
We present an image translation approach to generate augmented data for ...
research
∙
11/20/2018