research
∙
05/25/2023
Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training
Deep learning is experiencing a rise in foundation models that are expec...
share
research
∙
06/10/2022
Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models
Foundation models are becoming the dominant deep learning technologies. ...
research
∙
10/05/2021
S2 Reducer: High-Performance Sparse Communication to Accelerate Distributed Deep Learning
Distributed stochastic gradient descent (SGD) approach has been widely u...
research
∙
09/11/2018