research
∙
09/04/2023
Les Houches Lectures on Deep Learning at Large Infinite Width
These lectures, presented at the 2022 Les Houches Summer School on Stati...
share
research
∙
06/30/2021
The Evolution of Out-of-Distribution Robustness Throughout Fine-Tuning
Although machine learning models typically experience a drop in performa...
research
∙
02/12/2021
Explaining Neural Scaling Laws
The test loss of well-trained neural networks often follows precise powe...
research
∙
06/18/2020
Exact posterior distributions of wide Bayesian neural networks
Recent work has shown that the prior over functions induced by a deep Ba...
research
∙
06/18/2020
Infinite attention: NNGP and NTK for deep attention networks
There is a growing amount of literature on the relationship between wide...
research
∙
03/04/2020
The large learning rate phase of deep learning: the catapult mechanism
The choice of initial learning rate can have a profound effect on the pe...
research
∙
02/18/2019
Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
A longstanding goal in deep learning research has been to precisely char...
research
∙
10/11/2018
Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes
There is a previously identified equivalence between wide fully connecte...
research
∙
06/14/2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks
In recent years, state-of-the-art methods in computer vision have utiliz...
research
∙
02/23/2018
Sensitivity and Generalization in Neural Networks: an Empirical Study
In practice it is often found that large over-parameterized neural netwo...
research
∙
11/01/2017