research
∙
08/22/2023
Enhancing NeRF akin to Enhancing LLMs: Generalizable NeRF Transformer with Mixture-of-View-Experts
Cross-scene generalizable NeRF models, which can directly synthesize nov...
share
research
∙
08/19/2023
Robust Mixture-of-Expert Training for Convolutional Neural Networks
Sparsely-gated Mixture of Expert (MoE), an emerging deep model architect...
research
∙
06/25/2023
Enhancing Adversarial Training via Reweighting Optimization Trajectory
Despite the fact that adversarial training has become the de facto metho...
research
∙
06/24/2023
H_2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Large Language Models (LLMs), despite their recent impressive accomplish...
research
∙
06/18/2023
Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication
Graphs are omnipresent and GNNs are a powerful family of neural networks...
research
∙
06/18/2023
Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models
Large pre-trained transformers have been receiving explosive attention i...
research
∙
06/06/2023
The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter
Large pre-trained transformers are show-stealer in modern-day deep learn...
research
∙
03/03/2023
Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!
Sparse Neural Networks (SNNs) have received voluminous attention predomi...
research
∙
03/02/2023
Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers
Despite their remarkable achievement, gigantic transformers encounter si...
research
∙
02/28/2023
M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation
Learning to Optimize (L2O) has drawn increasing attention as it often re...
research
∙
02/22/2023
Learning to Generalize Provably in Learning to Optimize
Learning to optimize (L2O) has gained increasing popularity, which autom...
research
∙
12/06/2022
Attend Who is Weak: Pruning-assisted Medical Image Localization under Sophisticated and Implicit Imbalances
Deep neural networks (DNNs) have rapidly become a de facto choice for me...
research
∙
11/28/2022
You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets
Recent works have impressively demonstrated that there exists a subnetwo...
research
∙
10/26/2022
M^3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design
Multi-task learning (MTL) encapsulates multiple learned tasks in a singl...
research
∙
10/14/2022
Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again
Despite the enormous success of Graph Convolutional Networks (GCNs) in m...
research
∙
10/14/2022
A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinking
Large-scale graph training is a notoriously challenging problem for grap...
research
∙
10/08/2022
Advancing Model Pruning via Bi-level Optimization
The deployment constraints in practical applications necessitate the pru...
research
∙
10/07/2022
Augmentations in Hypergraph Contrastive Learning: Fabricated and Generative
This paper targets at improving the generalizability of hypergraph neura...
research
∙
09/15/2022
Can We Solve 3D Vision Tasks Starting from A 2D Vision Transformer?
Vision Transformers (ViTs) have proven to be effective, in solving 2D im...
research
∙
07/27/2022
Is Attention All NeRF Needs?
We present Generalizable NeRF Transformer (GNT), a pure, unified transfo...
research
∙
07/08/2022
Neural Implicit Dictionary via Mixture-of-Expert Training
Representing visual signals by coordinate-based deep fully-connected net...
research
∙
07/07/2022
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
Transformers have quickly shined in the computer vision world since the ...
research
∙
07/04/2022
Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentations
Neural Radiance Field (NeRF) regresses a neural parameterized scene by d...
research
∙
06/15/2022
Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning
Class-incremental learning (CIL) suffers from the notorious dilemma betw...
research
∙
06/15/2022
Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness
Certifiable robustness is a highly desirable property for adopting deep ...
research
∙
06/15/2022
Can pruning improve certified robustness of neural networks?
With the rapid development of deep learning, the sizes of neural network...
research
∙
05/24/2022
Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free
Trojan attacks threaten deep neural networks (DNNs) by poisoning them to...
research
∙
04/04/2022
APP: Anytime Progressive Pruning
With the latest advances in deep learning, there has been a lot of focus...
research
∙
03/13/2022
Symbolic Learning to Optimize: Towards Interpretability and Scalability
Recent studies on Learning to Optimize (L2O) suggest a promising path to...
research
∙
03/12/2022
Optimizer Amalgamation
Selecting an appropriate optimizer for a given problem is of major inter...
research
∙
03/12/2022
The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy
Vision transformers (ViTs) have gained increasing popularity as they are...
research
∙
03/09/2022
Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practice
Vision Transformer (ViT) has recently demonstrated promise in computer v...
research
∙
03/05/2022
Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance
Generative adversarial networks (GANs) have received an upsurging intere...
research
∙
02/09/2022
Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets
The lottery ticket hypothesis (LTH) has shown that dense models contain ...
research
∙
02/05/2022
The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training
Random pruning is arguably the most naive way to attain sparsity in neur...
research
∙
01/04/2022
Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations
Self-supervision is recently surging at its new frontier of graph learni...
research
∙
11/01/2021
Improving Contrastive Learning on Imbalanced Seed Data via Open-World Sampling
Contrastive learning approaches have achieved great success in learning ...
research
∙
10/30/2021
You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership
Despite tremendous success in many application scenarios, the training a...
research
∙
10/30/2021
DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models
Gigantic pre-trained models have become central to natural language proc...
research
∙
10/28/2021
CAP: Co-Adversarial Perturbation on Weights and Features for Improving Generalization of Graph Neural Networks
Despite the recent advances of graph neural networks (GNNs) in modeling ...
research
∙
10/09/2021
Towards Lifelong Learning of Multilingual Text-To-Speech Synthesis
This work presents a lifelong learning approach to train a multilingual ...
research
∙
10/07/2021
Universality of Deep Neural Network Lottery Tickets: A Renormalization Group Perspective
Foundational work on the Lottery Ticket Hypothesis has suggested an exci...
research
∙
08/24/2021
Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study
Training deep graph neural networks (GNNs) is notoriously hard. Besides ...
research
∙
07/16/2021
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference
Semantic segmentation for scene understanding is nowadays widely demande...
research
∙
07/01/2021
Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?
There have been long-standing controversies and inconsistencies over the...
research
∙
06/28/2021
FreeTickets: Accurate, Robust and Efficient Deep Ensemble by Training with Dynamic Sparsity
Recent works on sparse neural networks have demonstrated that it is poss...
research
∙
06/24/2021
Scalable Perception-Action-Communication Loops with Convolutional and Graph Neural Networks
In this paper, we present a perception-action-communication loop design ...
research
∙
06/19/2021
Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
Works on lottery ticket hypothesis (LTH) and single-shot network pruning...
research
∙
06/10/2021
Sparse and Imperceptible Adversarial Attack via a Homotopy Algorithm
Sparse adversarial attacks can fool deep neural networks (DNNs) by only ...
research
∙
06/08/2021