research
∙
06/21/2023
OBELISC: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Large multimodal models trained on natural documents, which interleave i...
share
research
∙
05/09/2023
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the r...
research
∙
03/07/2023
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
As language models grow ever larger, the need for large-scale high-quali...
research
∙
11/03/2022
Crosslingual Generalization through Multitask Finetuning
Multitask prompted finetuning (MTF) has been shown to help large languag...
research
∙
10/27/2022
What Language Model to Train if You Have One Million GPU Hours?
The crystallization of modeling methods around the Transformer architect...
research
∙
04/12/2022
What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?
Large pretrained Transformer language models have been shown to exhibit ...
research
∙
10/15/2021