research
∙
05/29/2023
Towards Constituting Mathematical Structures for Learning to Optimize
Learning to Optimize (L2O), a technique that utilizes machine learning t...
share
research
∙
07/07/2022
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
Transformers have quickly shined in the computer vision world since the ...
research
∙
02/05/2022
The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training
Random pruning is arguably the most naive way to attain sparsity in neur...
research
∙
12/18/2021
Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better
Federated learning (FL) enables distribution of machine learning workloa...
research
∙
10/29/2021
Hyperparameter Tuning is All You Need for LISTA
Learned Iterative Shrinkage-Thresholding Algorithm (LISTA) introduces th...
research
∙
07/01/2021
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
There have been long-standing controversies and inconsistencies over the...
research
∙
06/28/2021
FreeTickets: Accurate, Robust and Efficient Deep Ensemble by Training with Dynamic Sparsity
Recent works on sparse neural networks have demonstrated that it is poss...
research
∙
06/19/2021
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
Works on lottery ticket hypothesis (LTH) and single-shot network pruning...
research
∙
04/08/2021
A Design Space Study for LISTA and Beyond
In recent years, great success has been witnessed in building problem-sp...
research
∙
03/30/2021
The Elastic Lottery Ticket Hypothesis
Lottery Ticket Hypothesis raises keen attention to identifying sparse tr...
research
∙
03/23/2021
Learning to Optimize: A Primer and A Benchmark
Learning to optimize (L2O) is an emerging approach that leverages machin...
research
∙
01/04/2021
SmartDeal: Re-Modeling Deep Network Weights for Efficient Inference and Training
The record-breaking performance of deep neural networks (DNNs) comes wit...
research
∙
12/31/2020
EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
Deep, heavily overparameterized language models such as BERT, XLNet and ...
research
∙
10/24/2020
ShiftAddNet: A Hardware-Inspired Deep Network
Multiplication (e.g., convolution) is arguably a cornerstone of modern d...
research
∙
05/07/2020
SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation
We present SmartExchange, an algorithm-hardware co-design framework to t...
research
∙
03/04/2020
Safeguarded Learned Convex Optimization
Many applications require repeatedly solving a certain type of optimizat...
research
∙
10/29/2019
E2-Train: Energy-Efficient Deep Network Training with Data-, Model-, and Algorithm-Level Saving
Convolutional neural networks (CNNs) have been increasingly deployed to ...
research
∙
09/26/2019
Drawing early-bird tickets: Towards more efficient training of deep networks
(Frankle & Carbin, 2019) shows that there exist winning tickets (small b...
research
∙
05/14/2019
Plug-and-Play Methods Provably Converge with Properly Trained Denoisers
Plug-and-play (PnP) is a non-convex framework that integrates modern den...
research
∙
10/22/2018
Can We Gain More from Orthogonality Regularizations in Training Deep CNNs?
This paper seeks to answer the question: as the (near-) orthogonality of...
research
∙
08/29/2018