research
∙
09/09/2023
Stochastic Gradient Descent outperforms Gradient Descent in recovering a high-dimensional signal in a glassy energy landscape
Stochastic Gradient Descent (SGD) is an out-of-equilibrium algorithm use...
share
research
∙
12/20/2021
The effective noise of Stochastic Gradient Descent
Stochastic Gradient Descent (SGD) is the workhorse algorithm of deep lea...
research
∙
03/08/2021
Stochasticity helps to navigate rough landscapes: comparing gradient-descent-based algorithms in the phase retrieval problem
In this paper we investigate how gradient-based algorithms such as gradi...
research
∙
02/23/2021
Just a Momentum: Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problem
When optimizing over loss functions it is common practice to use momentu...
research
∙
06/12/2020
Complex Dynamics in Simple Neural Networks: Understanding Gradient Flow in Phase Retrieval
Despite the widespread use of gradient-based algorithms for optimizing h...
research
∙
06/10/2020
Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classification
We analyze in a closed form the learning dynamics of stochastic gradient...
research
∙
02/01/2019
Passed & Spurious: analysing descent algorithms and local minima in spiked matrix-tensor model
In this work we analyse quantitatively the interplay between the loss la...
research
∙
12/21/2018
Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
Gradient-descent-based algorithms and their stochastic versions have wid...
research
∙
07/03/2018
Approximate Survey Propagation for Statistical Inference
Approximate message passing algorithm enjoyed considerable attention in ...
research
∙
05/15/2018