research
∙
06/20/2023
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Transformer models have achieved remarkable results in various natural l...
share
research
∙
03/18/2023
Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Fine-tuning large pre-trained language models on downstream tasks has be...
research
∙
10/04/2022
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
Layer-wise distillation is a powerful tool to compress large models (i.e...
research
∙
06/25/2022
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in va...
research
∙
04/15/2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
Pre-trained language models have demonstrated superior performance in va...
research
∙
03/01/2021
A Biased Graph Neural Network Sampler with Near-Optimal Regret
Graph neural networks (GNN) have recently emerged as a vehicle for apply...
research
∙
09/29/2018