research
∙
09/15/2023
Scaling Laws for Sparsely-Connected Foundation Models
We explore the impact of parameter sparsity on the scaling behavior of T...
share
research
∙
05/03/2023
Dynamic Sparse Training with Structured Sparsity
DST methods achieve state-of-the-art results in sparse neural network tr...
research
∙
04/27/2023
JaxPruner: A concise library for sparsity research
This paper introduces JaxPruner, an open-source JAX-based pruning and sp...
research
∙
02/24/2023
The Dormant Neuron Phenomenon in Deep Reinforcement Learning
In this work we identify the dormant neuron phenomenon in deep reinforce...
research
∙
09/15/2022
Training Recipe for N:M Structured Sparsity with Decaying Pruning Mask
Sparsity has become one of the promising methods to compress and acceler...
research
∙
06/17/2022
The State of Sparse Training in Deep Reinforcement Learning
The use of sparse neural networks has seen rapid growth in recent years,...
research
∙
01/13/2022
GradMax: Growing Neural Networks using Gradient Information
The architecture and the parameters of neural networks are often optimiz...
research
∙
01/10/2022
Head2Toe: Utilizing Intermediate Representations for Better Transfer Learning
Transfer-learning methods aim to improve performance in a data-scarce ta...
research
∙
04/06/2021
Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark
Meta and transfer learning are two successful families of approaches to ...
research
∙
10/07/2020
Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win
Sparse Neural Networks (NNs) can match the generalization of dense NNs u...
research
∙
06/12/2020
A Practical Sparse Approximation for Real Time Recurrent Learning
Current methods for training recurrent neural networks are based on back...
research
∙
11/25/2019
Rigging the Lottery: Making All Tickets Winners
Sparse neural networks have been shown to be more parameter and compute ...
research
∙
07/01/2019
Natural Language Understanding with the Quora Question Pairs Dataset
This paper explores the task Natural Language Understanding (NLU) by loo...
research
∙
06/25/2019
The Difficulty of Training Sparse Neural Networks
We investigate the difficulties of training sparse neural networks and m...
research
∙
06/15/2018