research
∙
08/25/2023
Six Lectures on Linearized Neural Networks
In these six lectures, we examine what can be learnt about the behavior ...
share
research
∙
05/18/2023
Sampling, Diffusions, and Stochastic Localization
Diffusions are a successful technique to sample from high-dimensional di...
research
∙
04/22/2023
Posterior Sampling from the Spiked Models via Diffusion Processes
Sampling from the posterior is a key technical problem in Bayesian stati...
research
∙
02/28/2023
Learning time-scales in two-layers neural networks
Gradient-based learning in multi-layer neural networks displays a number...
research
∙
02/20/2023
Compressing Tabular Data via Latent Variable Estimation
Data used for analytics and machine learning often take the form of tabl...
research
∙
12/14/2022
Equivalence of Approximate Message Passing and Low-Degree Polynomials in Rank-One Matrix Estimation
We consider the problem of estimating an unknown parameter vector θ∈ℝ^n,...
research
∙
11/01/2022
Fundamental Limits of Low-Rank Matrix Estimation with Diverging Aspect Ratios
We consider the problem of estimating the factors of a low-rank n × d ma...
research
∙
10/16/2022
Dimension free ridge regression
Random matrix theory has become a widely useful tool in high-dimensional...
research
∙
06/14/2022
Overparametrized linear dimensionality reductions: From projection pursuit to two-layer neural networks
Given a cloud of n data points in ℝ^d, consider all projections onto m-d...
research
∙
04/06/2022
A Short Tutorial on Mean-Field Spin Glass Techniques for Non-Physicists
This tutorial is based on lecture notes written for a class taught in th...
research
∙
03/31/2022
Adversarial Examples in Random Neural Networks with General Activations
A substantial body of empirical work documents the lack of robustness in...
research
∙
03/10/2022
Sampling from the Sherrington-Kirkpatrick Gibbs measure via algorithmic stochastic localization
We consider the Sherrington-Kirkpatrick model of spin glasses at high-te...
research
∙
02/17/2022
Universality of empirical risk minimization
Consider supervised learning from i.i.d. samples { x_i,y_i}_i≤ n where x...
research
∙
01/13/2022
Statistically Optimal First Order Algorithms: A Proof via Orthogonalization
We consider a class of statistical estimation problems in which we are g...
research
∙
12/14/2021
The high-dimensional asymptotics of first order methods with random data
We study a class of deterministic flows in ℝ^d× k, parametrized by a ran...
research
∙
11/12/2021
Local algorithms for Maximum Cut and Minimum Bisection on locally treelike regular graphs of large degree
Given a graph G of degree k over n vertices, we consider the problem of ...
research
∙
10/28/2021
Tractability from overparametrization: The example of the negative perceptron
In the negative perceptron problem we are given n data points ( x_i,y_i)...
research
∙
09/02/2021
An Information-Theoretic View of Stochastic Localization
Given a probability measure μ over ℝ^n, it is often useful to approximat...
research
∙
07/29/2021
CAD: Debiasing the Lasso with inaccurate covariate model
We consider the problem of estimating a low-dimensional parameter in hig...
research
∙
06/09/2021
Streaming Belief Propagation for Community Detection
The community detection problem requires to cluster the nodes of a netwo...
research
∙
03/30/2021
Minimum complexity interpolation in random features models
Despite their many appealing properties, kernel methods are heavily affe...
research
∙
03/16/2021
Deep learning: a statistical viewpoint
The remarkable practical success of deep learning has revealed some majo...
research
∙
02/25/2021
Learning with invariances in random features and kernel models
A number of machine learning tasks entail a high degree of invariance: t...
research
∙
01/26/2021
Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration
Consider the classical supervised learning problem: we are given data (y...
research
∙
11/06/2020
Underspecification Presents Challenges for Credibility in Modern Machine Learning
ML models often exhibit unexpectedly poor behavior when they are deploye...
research
∙
07/27/2020
The Lasso with general Gaussian designs with applications to hypothesis testing
The Lasso is a method for high-dimensional regression, which is now comm...
research
∙
07/25/2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training
Modern neural networks are often operated in a strongly overparametrized...
research
∙
06/24/2020
When Do Neural Networks Outperform Kernel Methods?
For a certain scaling of the initialization of stochastic gradient desce...
research
∙
02/28/2020
The estimation error of general first order methods
Modern large-scale statistical models require to estimate thousands to m...
research
∙
01/24/2020
Imputation for High-Dimensional Linear Regression
We study high-dimensional regression with missing entries in the covaria...
research
∙
11/05/2019
The generalization error of max-margin linear classifiers: High-dimensional asymptotics in the overparametrized regime
Modern machine learning models are often so complex that they achieve va...
research
∙
08/14/2019
The generalization error of random features regression: Precise asymptotics and double descent curve
Deep learning methods operate in regimes that defy the traditional stati...
research
∙
06/21/2019
Limitations of Lazy Training of Two-layers Neural Networks
We study the supervised learning problem under either of the following t...
research
∙
04/27/2019
Linearized two-layers neural networks in high dimension
We consider the problem of learning an unknown function f_ on the d-dime...
research
∙
04/05/2019
On the computational tractability of statistical estimation on amenable graphs
We consider the problem of estimating a vector of discrete variables (θ_...
research
∙
03/25/2019
Fundamental Barriers to High-Dimensional Regression with Convex Penalties
In high-dimensional regression, we attempt to estimate a parameter vecto...
research
∙
03/19/2019
Surprises in High-Dimensional Ridgeless Least Squares Interpolation
Interpolators -- estimators that achieve zero training error -- have att...
research
∙
02/16/2019
Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit
We consider learning two layer neural networks using stochastic gradient...
research
∙
01/05/2019
Analysis of a Two-Layer Neural Network via Displacement Convexity
Fitting a function by using linear combinations of a large number N of `...
research
∙
11/03/2018
The distribution of the Lasso: Uniform control over sparse balls and adaptive parameter tuning
The Lasso is a popular regression method for high-dimensional problems i...
research
∙
10/06/2018
Adapting to Unknown Noise Distribution in Matrix Denoising
We consider the problem of estimating an unknown matrix ∈^m× n, from obs...
research
∙
08/23/2018
TAP free energy, spin glasses, and variational inference
We consider the Sherrington-Kirkpatrick model of spin glasses with ferro...
research
∙
07/23/2018
Contextual Stochastic Block Models
We provide the first information theoretic tight analysis for inference ...
research
∙
04/18/2018
A Mean Field View of the Landscape of Two-Layers Neural Networks
Multi-layer neural networks are among the most powerful models in machin...
research
∙
04/14/2018
The threshold for SDP-refutation of random regular NAE-3SAT
Unlike its cousin 3SAT, the NAE-3SAT (not-all-equal-3SAT) problem has th...
research
∙
02/20/2018
On the Connection Between Learning Two-Layers Neural Networks and Tensor Decomposition
We establish connections between the problem of learning a two-layers ne...
research
∙
02/02/2018
An Instability in Variational Inference for Topic Models
Topic models are Bayesian models that are frequently used to capture the...
research
∙
11/15/2017
The landscape of the spiked tensor model
We consider the problem of estimating a large rank-one tensor u^⊗ k∈( R...
research
∙
11/06/2017
Estimation of Low-Rank Matrices via Approximate Message Passing
Consider the problem of estimating a low-rank symmetric matrix when its ...
research
∙
09/19/2017