research
∙
06/13/2023
SqueezeLLM: Dense-and-Sparse Quantization
Generative Large Language Models (LLMs) have demonstrated remarkable res...
share
research
∙
06/01/2023
Towards Foundation Models for Scientific Machine Learning: Characterizing Scaling and Transfer Behavior
Pre-trained machine learning (ML) models have shown great performance fo...
research
∙
02/27/2023
Full Stack Optimization of Transformer Inference: a Survey
Recent advances in state-of-the-art DNN architecture design have been mo...
research
∙
02/15/2023
Big Little Transformer Decoder
The recent emergence of Large Language Models based on the Transformer a...
research
∙
07/08/2022
Adaptive Self-supervision Algorithms for Physics-informed Neural Networks
Physics-informed neural networks (PINNs) incorporate physical knowledge ...
research
∙
06/02/2022
Squeezeformer: An Efficient Transformer for Automatic Speech Recognition
The recently proposed Conformer model has become the de facto backbone m...
research
∙
09/02/2021
Characterizing possible failure modes in physics-informed neural networks
Recent work in scientific machine learning has developed so-called physi...
research
∙
07/02/2021
Learned Token Pruning for Transformers
A major challenge in deploying transformer models is their prohibitive i...
research
∙
03/31/2021
Q-ASR: Integer-only Zero-shot Quantization for Efficient Speech Recognition
End-to-end neural network models achieve improved performance on various...
research
∙
03/25/2021
A Survey of Quantization Methods for Efficient Neural Network Inference
As soon as abstract mathematical computations were adapted to computatio...
research
∙
01/22/2021
Hessian-Aware Pruning and Optimal Neural Implant
Pruning is an effective method to reduce the memory footprint and FLOPs ...
research
∙
01/05/2021
I-BERT: Integer-only BERT Quantization
Transformer based models, like BERT and RoBERTa, have achieved state-of-...
research
∙
11/20/2020
HAWQV3: Dyadic Neural Network Quantization
Quantization is one of the key techniques used to make Neural Networks (...
research
∙
07/09/2020
Boundary thickness and robustness in learning models
Robustness of machine learning models to various adversarial and non-adv...
research
∙
06/01/2020
ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
We introduce AdaHessian, a second order stochastic optimization algorith...
research
∙
03/17/2020
Rethinking Batch Normalization in Transformers
The standard normalization method for neural network (NN) models used in...
research
∙
01/01/2020
ZeroQ: A Novel Zero Shot Quantization Framework
Quantization is a promising approach for reducing the inference time and...
research
∙
12/16/2019
PyHessian: Neural Networks Through the Lens of the Hessian
We present PyHessian, a new scalable framework that enables fast computa...
research
∙
11/10/2019
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks
Quantization is an effective method for reducing memory footprint and in...
research
∙
10/07/2019
Checkmate: Breaking the Memory Wall with Optimal Tensor Rematerialization
Modern neural networks are increasingly bottlenecked by the limited capa...
research
∙
09/12/2019
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Transformer based architectures have become de-facto models used for a r...
research
∙
06/10/2019
ANODEV2: A Coupled Neural ODE Evolution Framework
It has been observed that residual networks can be viewed as the explici...
research
∙
03/14/2019
Inefficiency of K-FAC for Large Batch Size Training
In stochastic optimization, large batch training can leverage parallel r...
research
∙
02/27/2019
ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs
Residual neural networks can be viewed as the forward Euler discretizati...
research
∙
12/16/2018
Trust Region Based Adversarial Attack on Neural Networks
Deep Neural Networks are quite vulnerable to adversarial perturbations. ...
research
∙
12/04/2018
Parameter Re-Initialization through Cyclical Batch Size Schedules
Optimal parameter initialization remains a crucial problem for neural ne...
research
∙
11/30/2018
On the Computational Inefficiency of Large Batch Sizes for Stochastic Gradient Descent
Increasing the mini-batch size for stochastic gradient descent offers si...
research
∙
11/05/2018
Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
Gliomas are the most common primary brain malignancies, with different d...
research
∙
10/11/2018
A Novel Domain Adaptation Framework for Medical Image Segmentation
We propose a segmentation framework that uses deep neural networks and i...
research
∙
10/02/2018
Large batch size training of neural networks with adversarial training and second-order information
Stochastic Gradient Descent (SGD) methods using randomly selected batche...
research
∙
08/13/2018
CLAIRE: A distributed-memory solver for constrained large deformation diffeomorphic image registration
We introduce CLAIRE, a distributed-memory algorithm and software for sol...
research
∙
04/20/2018
Co-Design of Deep Neural Nets and Neural Net Accelerators for Embedded Vision Applications
Deep Learning is arguably the most rapidly evolving research area in rec...
research
∙
03/23/2018
SqueezeNext: Hardware-Aware Neural Network Design
One of the main barriers for deploying neural networks on embedded syste...
research
∙
02/28/2018
PDE-constrained optimization in medical image analysis
PDE-constrained optimization problems find many applications in medical ...
research
∙
02/22/2018
Hessian-based Analysis of Large Batch Training and Robustness to Adversaries
Large batch size training of Neural Networks has been shown to incur acc...
research
∙
12/12/2017
Integrated Model, Batch and Domain Parallelism in Training Neural Networks
We propose a new integrated method of exploiting model, batch and domain...
research
∙
12/12/2017
Integrated Model and Data Parallelism in Training Neural Networks
We propose a new integrated method of exploiting both model and data par...
research
∙
08/11/2016