research
∙
08/08/2023
Quantization Aware Factorization for Deep Neural Network Compression
Tensor decomposition of convolutional and fully-connected layers is an e...
share
research
∙
07/03/2023
Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch
We propose Rockmate to control the memory requirements when training PyT...
research
∙
06/05/2023
Efficient GPT Model Pre-training using Tensor Train Matrix Representation
Large-scale transformer models have shown remarkable performance in lang...
research
∙
02/21/2022
Survey on Large Scale Neural Network Training
Modern Deep Neural Networks (DNNs) require significant memory to store w...
research
∙
02/01/2022
Few-Bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction
Memory footprint is one of the main limiting factors for large neural ne...
research
∙
01/31/2022
Memory-Efficient Backpropagation through Large Linear Layers
In modern neural networks like Transformers, linear layers require signi...
research
∙
03/15/2021
Meta-Solver for Neural Ordinary Differential Equations
A conventional approach to train neural ordinary differential equations ...
research
∙
08/12/2020
Stable Low-rank Tensor Decomposition for Compression of Convolutional Neural Network
Most state of the art deep neural networks are overparameterized and exh...
research
∙
04/20/2020
Towards Understanding Normalization in Neural ODEs
Normalization is an important and vastly investigated technique in deep ...
research
∙
03/11/2020
Interpolated Adjoint Method for Neural ODEs
In this paper, we propose a method, which allows us to alleviate or comp...
research
∙
10/29/2019
Active Subspace of Neural Networks: Structural Analysis and Universal Attacks
Active subspace is a model reduction method widely used in the uncertain...
research
∙
10/15/2019
Reduced-Order Modeling of Deep Neural Networks
We introduce a new method for speeding up the inference of deep neural n...
research
∙
03/24/2019