research
∙
05/30/2023
Bottleneck Structure in Learned Features: Low-Dimension vs Regularity Tradeoff
Previous work has shown that DNNs with large depth L and L_2-regularizat...
share
research
∙
05/25/2023
Implicit bias of SGD in L_2-regularized linear DNNs: One-way jumps from high to low rank
The L_2-regularized loss of Deep Linear Networks (DLNs) with more than o...
research
∙
09/29/2022
Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions
We show that the representation cost of fully connected neural networks ...
research
∙
05/31/2022
Feature Learning in L_2-regularized DNNs: Attraction/Repulsion and Sparsity
We study the loss surface of DNNs with L_2 regularization. We show that ...
research
∙
11/06/2021
Understanding Layer-wise Contributions in Deep Neural Networks through Spectral Analysis
Spectral analysis is a powerful tool, decomposing any function into simp...
research
∙
06/30/2021
Deep Linear Networks Dynamics: Low-Rank Biases Induced by Initialization Scale and L2 Regularization
For deep linear networks (DLN), various hyperparameters alter the dynami...
research
∙
06/10/2021
DNN-Based Topology Optimisation: Spatial Invariance and Neural Tangent Kernel
We study the SIMP method with a density field generated by a fully-conne...
research
∙
05/25/2021
Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances
We study how permutation symmetries in overparameterized multi-layer neu...
research
∙
06/17/2020
Kernel Alignment Risk Estimator: Risk Prediction from Training Data
We study the risk (i.e. generalization error) of Kernel Ridge Regression...
research
∙
02/19/2020
Implicit Regularization of Random Feature Models
Random Feature (RF) models are used as efficient parametric approximatio...
research
∙
10/01/2019
The asymptotic spectrum of the Hessian of DNN throughout training
The dynamics of DNNs during gradient descent is described by the so-call...
research
∙
07/11/2019
Freeze and Chaos for DNNs: an NTK view of Batch Normalization, Checkerboard and Boundary Effects
In this paper, we analyze a number of architectural features of Deep Neu...
research
∙
06/19/2019
Disentangling feature and lazy learning in deep neural networks: an empirical study
Two distinct limits for deep learning as the net width h→∞ have been pro...
research
∙
01/06/2019
Scaling description of generalization with number of parameters in deep learning
We provide a description for the evolution of the generalization perform...
research
∙
06/20/2018