research
∙
08/18/2023
Towards Understanding the Generalizability of Delayed Stochastic Gradient Descent
Stochastic gradient descent (SGD) performed in an asynchronous manner pl...
share
research
∙
05/25/2023
Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training
Deep learning is experiencing a rise in foundation models that are expec...
research
∙
06/10/2022
Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models
Foundation models are becoming the dominant deep learning technologies. ...
research
∙
10/18/2021