research
∙
08/03/2023
Accurate Neural Network Pruning Requires Rethinking Sparse Optimization
Obtaining versions of deep neural networks that are both highly-accurate...
share
research
∙
06/09/2023
Error Feedback Can Accurately Compress Preconditioners
Leveraging second-order information at the scale of deep networks is one...
research
∙
03/25/2023
Vision Models Can Be Efficiently Specialized via Few-Shot Task-Aware Compression
Recent vision architectures and self-supervised training methods enable ...
research
∙
02/09/2023
SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks
We provide a new efficient version of the backpropagation algorithm, spe...
research
∙
02/07/2023
ZipLM: Hardware-Aware Structured Pruning of Language Models
The breakthrough performance of large language models (LLMs) comes with ...
research
∙
10/14/2022
oViT: An Accurate Second-Order Pruning Framework for Vision Transformers
Models from the Vision Transformer (ViT) family have recently provided b...
research
∙
10/12/2022
GMP*: Well-Tuned Global Magnitude Pruning Can Outperform Most BERT-Pruning Methods
We revisit the performance of the classic gradual magnitude pruning (GMP...
research
∙
09/08/2022
Vision for Bosnia and Herzegovina in Artificial Intelligence Age: Global Trends, Potential Opportunities, Selected Use-cases and Realistic Goals
Artificial Intelligence (AI) is one of the most promising technologies o...
research
∙
03/14/2022
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models
Pre-trained Transformer-based language models have become a key building...
research
∙
07/07/2021