research
∙
09/19/2023
On the different regimes of Stochastic Gradient Descent
Modern deep networks are trained with stochastic gradient descent (SGD) ...
share
research
∙
07/05/2023
How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model
Learning generic high-dimensional tasks is notably hard, as it requires ...
research
∙
01/31/2023
Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning
Understanding when the noise in stochastic gradient descent (SGD) affect...
research
∙
10/04/2022
How deep convolutional neural networks lose spatial information with training
A central question of machine learning is how deep nets manage to learn ...
research
∙
08/01/2022
How Wide Convolutional Neural Networks Learn Hierarchical Tasks
Despite their success, understanding how convolutional neural networks (...
research
∙
06/24/2022
Learning sparse features can lead to overfitting in neural networks
It is widely believed that the success of deep networks lies in their ab...
research
∙
02/07/2022
Failure and success of the spectral bias prediction for Kernel Ridge Regression: the case of low-dimensional data
Recently, several theories including the replica method made predictions...
research
∙
06/16/2021
How memory architecture affects performance and learning in simple POMDPs
Reinforcement learning is made much more complex when the agent's observ...
research
∙
06/16/2021
Locality defeats the curse of dimensionality in convolutional teacher-student scenarios
Convolutional neural networks perform a local and translationally-invari...
research
∙
05/06/2021
Relative stability toward diffeomorphisms in deep nets indicates performance
Understanding why deep nets can classify data in large dimensions remain...
research
∙
12/30/2020
Perspective: A Phase Diagram for Deep Learning unifying Jamming, Feature Learning and Lazy Training
Deep learning algorithms are responsible for a technological revolution ...
research
∙
07/22/2020
Compressing invariant manifolds in neural nets
We study how neural networks compress uninformative input space in model...
research
∙
06/17/2020
How isotropic kernels learn simple invariants
We investigate how the training curve of isotropic kernel methods depend...
research
∙
06/19/2019
Disentangling feature and lazy learning in deep neural networks: an empirical study
Two distinct limits for deep learning as the net width h→∞ have been pro...
research
∙
05/26/2019
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm
How many training data are needed to learn a supervised task? It is ofte...
research
∙
01/06/2019
Scaling description of generalization with number of parameters in deep learning
We provide a description for the evolution of the generalization perform...
research
∙
10/22/2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization
We argue that in fully-connected networks a phase transition delimits th...
research
∙
09/25/2018