research
∙
01/14/2021
Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration
Adam is one of the most influential adaptive stochastic algorithms for t...
share
research
∙
11/23/2018
A Sufficient Condition for Convergences of Adam and RMSProp
Adam and RMSProp, as two of the most influential adaptive stochastic alg...
research
∙
08/10/2018
Weighted AdaGrad with Unified Momentum
Integrating adaptive learning rate and momentum techniques into SGD lead...
research
∙
08/10/2018
On the Convergence of Weighted AdaGrad with Momentum for Training Deep Neural Networks
Adaptive stochastic gradient descent methods, such as AdaGrad, RMSProp, ...
research
∙
08/10/2018