research
∙
04/22/2023
Collie: Finding Performance Anomalies in RDMA Subsystems
High-speed RDMA networks are getting rapidly adopted in the industry for...
share
research
∙
10/31/2022
Lita: Accelerating Distributed Training of Sparsely Activated Models
Scaling model parameters usually improves model quality, but at the pric...
research
∙
10/06/2022
ByteTransformer: A High-Performance Transformer Boosted for Variable-Length Inputs
Transformer is the cornerstone model of Natural Language Processing (NLP...
research
∙
05/28/2022
ByteComp: Revisiting Gradient Compression in Distributed Training
Gradient compression (GC) is a promising approach to addressing the comm...
research
∙
05/05/2022
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
Distributed training using multiple devices (e.g., GPUs) has been widely...
research
∙
02/16/2022
Aryl: An Elastic Cluster Scheduler for Deep Learning
Companies build separate training and inference GPU clusters for deep le...
research
∙
12/16/2021
BGL: GPU-Efficient GNN Training by Optimizing Graph Data I/O and Preprocessing
Graph neural networks (GNNs) have extended the success of deep neural ne...
research
∙
10/25/2021
Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performance
Today's auto-tuners (e.g., AutoTVM, Ansor) generate efficient tensor pro...
research
∙
09/18/2021
Serving DNN Models with Multi-Instance GPUs: A Case of the Reconfigurable Machine Scheduling Problem
Multi-Instance GPU (MIG) is a new feature introduced by NVIDIA A100 GPUs...
research
∙
07/19/2021
DeepCC: Bridging the Gap Between Congestion Control and Applications via Multi-Objective Optimization
The increasingly complicated and diverse applications have distinct netw...
research
∙
05/22/2021
AutoLRS: Automatic Learning-Rate Schedule by Bayesian Optimization on the Fly
The learning rate (LR) schedule is one of the most important hyper-param...
research
∙
02/20/2018