research
∙
07/20/2023
Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case
Adam is a commonly used stochastic optimization algorithm in machine lea...
share
research
∙
05/09/2023
UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization
Adam-type algorithms have become a preferred choice for optimisation in ...
research
∙
05/30/2022
Last-iterate convergence analysis of stochastic momentum methods for neural networks
The stochastic momentum method is a commonly used acceleration technique...
research
∙
06/12/2021
Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descent
The plain stochastic gradient descent and momentum stochastic gradient d...
research
∙
06/12/2021