research
∙
06/05/2023
SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression
Recent advances in large language model (LLM) pretraining have led to hi...
share
research
∙
01/27/2023
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient
Many deep learning applications benefit from using large models with bil...
research
∙
09/02/2022
Petals: Collaborative Inference and Fine-tuning of Large Models
Many NLP tasks benefit from using large language models (LLMs) that ofte...
research
∙
07/07/2022
Training Transformers Together
The infrastructure necessary for training state-of-the-art models is bec...
research
∙
06/21/2021
Secure Distributed Training at Scale
Some of the hardest problems in deep learning can be solved with the com...
research
∙
06/18/2021