research
∙
08/08/2023
Continual Pre-Training of Large Language Models: How to (re)warm your model?
Large language models (LLMs) are routinely pre-trained on billions of to...
share
research
∙
04/21/2023
Emergent and Predictable Memorization in Large Language Models
Memorization, or the tendency of large language models (LLMs) to output ...
research
∙
04/03/2023
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
How do large language models (LLMs) develop and evolve over the course o...
research
∙
03/15/2023
MCR-DL: Mix-and-Match Communication Runtime for Deep Learning
In recent years, the training requirements of many state-of-the-art Deep...
research
∙
04/14/2022
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
We introduce GPT-NeoX-20B, a 20 billion parameter autoregressive languag...
research
∙
09/17/2021
Cross-layer Visualization and Profiling of Network and I/O Communication for HPC Clusters
Understanding and visualizing the full-stack performance trade-offs and ...
research
∙
11/12/2019