research
∙
09/12/2023
The Grand Illusion: The Myth of Software Portability and Implications for ML Progress
Pushing the boundaries of machine learning often requires exploring diff...
share
research
∙
09/11/2023
Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning
The Mixture of Experts (MoE) is a widely known neural architecture where...
research
∙
09/08/2023
When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale
Large volumes of text data have contributed significantly to the develop...
research
∙
07/06/2023
Frontier AI Regulation: Managing Emerging Risks to Public Safety
Advanced AI models hold the promise of tremendous benefits for humanity,...
research
∙
06/09/2023
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Generative AI systems across modalities, ranging from text, image, audio...
research
∙
05/30/2023
Intriguing Properties of Quantization at Scale
Emergent properties have been widely adopted as a term to describe behav...
research
∙
04/24/2023
On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research
Perception of toxicity evolves over time and often differs between geogr...
research
∙
03/01/2023
FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling
Ensembling independent deep neural networks (DNNs) is a simple and effec...
research
∙
11/04/2022
Intriguing Properties of Compression on Multilingual Models
Multilingual models are often particularly dependent on scaling to gener...
research
∙
10/26/2022
Large language models are not zero-shot communicators
Despite widespread use of LLMs as conversational agents, evaluations of ...
research
∙
09/20/2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics
Modern machine learning research relies on relatively few carefully cura...
research
∙
08/31/2022
Efficient Methods for Natural Language Processing: A Survey
Getting the most out of limited resources allows advances in natural lan...
research
∙
07/01/2022
Studying the impact of magnitude pruning on contrastive learning methods
We study the impact of different pruning techniques on the representatio...
research
∙
06/13/2022
Robust Distillation for Worst-class Performance
Knowledge distillation has proven to be an effective technique in improv...
research
∙
01/14/2022
When less is more: Simplifying inputs aids neural network understanding
How do neural network image classifiers respond to simpler and simpler i...
research
∙
10/06/2021
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
A "bigger is better" explosion in the number of parameters in deep neura...
research
∙
07/27/2021
A Tale Of Two Long Tails
As machine learning models are increasingly employed to assist human dec...
research
∙
07/16/2021
When does loss-based prioritization fail?
Not all examples are created equal, but standard deep neural network tra...
research
∙
06/22/2021
Randomness In Neural Network Training: Characterizing The Impact of Tooling
The quest for determinism in machine learning has disproportionately foc...
research
∙
02/02/2021
Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network Optimization
Training sparse networks to converge to the same performance as dense ne...
research
∙
10/06/2020
Characterising Bias in Compressed Models
The popularity and widespread use of pruning and quantization is driven ...
research
∙
09/14/2020
The Hardware Lottery
Hardware, systems and algorithms research communities have historically ...
research
∙
08/26/2020
Estimating Example Difficulty using Variance of Gradients
In machine learning, a question of great interest is understanding what ...
research
∙
04/15/2020
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
With the recent wave of progress in artificial intelligence (AI) has com...
research
∙
11/13/2019
Selective Brain Damage: Measuring the Disparate Impact of Model Pruning
Neural network pruning techniques have demonstrated it is possible to re...
research
∙
02/25/2019
The State of Sparsity in Deep Neural Networks
We rigorously evaluate three state-of-the-art techniques for inducing sp...
research
∙
06/28/2018
Evaluating Feature Importance Estimates
Estimating the influence of a given feature to a model prediction is cha...
research
∙
11/02/2017