research
∙
06/06/2023
Generalized Pseudospectral Shattering and Inverse-Free Matrix Pencil Diagonalization
We present a randomized, inverse-free algorithm for producing an approxi...
share
research
∙
01/29/2023
Fast Exact Leverage Score Sampling from Khatri-Rao Products with Applications to Tensor Decomposition
We present a data structure to randomly sample rows from the Khatri-Rao ...
research
∙
10/11/2022
Distributed-Memory Randomized Algorithms for Sparse Tensor CP Decomposition
Low-rank Candecomp / PARAFAC (CP) Decomposition is a powerful tool for t...
research
∙
07/19/2022
Proposed Consistent Exception Handling for the BLAS and LAPACK
Numerical exceptions, which may be caused by overflow, operations like d...
research
∙
04/18/2022
Communication Bounds for Convolutional Neural Networks
Convolutional neural networks (CNNs) are important in a wide variety of ...
research
∙
03/15/2022
Distributed-Memory Sparse Kernels for Machine Learning
Sampled Dense Times Dense Matrix Multiplication (SDDMM) and Sparse Times...
research
∙
05/05/2021
CoSA: Scheduling by Constrained Optimization for Spatial Accelerators
Recent advances in Deep Neural Networks (DNNs) have led to active develo...
research
∙
11/16/2020
Avoiding Communication in Logistic Regression
Stochastic gradient descent (SGD) is one of the most widely used optimiz...
research
∙
10/30/2020
Training EfficientNets at Supercomputer Scale: 83 Accuracy in One Hour
EfficientNets are a family of state-of-the-art image classification mode...
research
∙
06/15/2020
The Limit of the Batch Size
Large-batch training is an efficient approach for current distributed de...
research
∙
02/28/2020
Communication-Optimal Tilings for Projective Nested Loops with Arbitrary Bounds
Reducing communication - either between levels of a memory hierarchy or ...
research
∙
11/20/2019
Auto-Precision Scaling for Distributed Deep Learning
In recent years, large-batch optimization is becoming the key of distrib...
research
∙
10/01/2019
An improved analysis and unified perspective on deterministic and randomized low rank matrix approximations
We introduce a Generalized LU-Factorization (GLU) for low-rank matrix ap...
research
∙
09/14/2019
A Generalized Randomized Rank-Revealing Factorization
We introduce a Generalized Randomized QR-decomposition that may be appli...
research
∙
04/01/2019
Reducing BERT Pre-Training Time from 3 Days to 76 Minutes
Large-batch training is key to speeding up deep neural network training ...
research
∙
01/24/2019
Large-Batch Training for LSTM and Beyond
Large-batch training approaches have enabled researchers to utilize larg...
research
∙
05/14/2018
A 3D Parallel Algorithm for QR Decomposition
Interprocessor communication often dominates the runtime of large matrix...
research
∙
05/01/2018
Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems
We propose two new methods to address the weak scaling problems of KRR: ...
research
∙
02/19/2018
Communication-Optimal Convolutional Neural Nets
Efficiently executing convolutional neural nets (CNNs) is important in m...
research
∙
12/17/2017
Avoiding Synchronization in First-Order Methods for Sparse Convex Optimization
Parallel computing has played an important role in speeding up convex op...
research
∙
10/24/2017
Avoiding Communication in Proximal Methods for Convex Optimization Problems
The fast iterative soft thresholding algorithm (FISTA) is used to solve ...
research
∙
09/14/2017
ImageNet Training in Minutes
Finishing 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIA M40 G...
research
∙
07/14/2017