research
∙
05/19/2023
Beyond Exponential Graph: Communication-Efficient Topologies for Decentralized Learning via Finite-time Convergence
Decentralized learning has recently been attracting increasing attention...
share
research
∙
09/30/2022
Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data
SGD with momentum acceleration is one of the key components for improvin...
research
∙
05/23/2022
Theoretical Analysis of Primal-Dual Algorithm for Non-Convex Stochastic Decentralized Optimization
In recent years, decentralized learning has emerged as a powerful tool n...
research
∙
05/08/2022
Communication Compression for Decentralized Learning with Operator Splitting Methods
In decentralized learning, operator splitting methods using a primal-dua...
research
∙
03/24/2022
On Exploiting Layerwise Gradient Statistics for Effective Training of Deep Neural Networks
Adam and AdaBelief compute and make use of elementwise adaptive stepsize...
research
∙
07/19/2021
Revisiting the Primal-Dual Method of Multipliers for Optimisation over Centralised Networks
The primal-dual method of multipliers (PDMM) was originally designed for...
research
∙
11/21/2019
Approximated Orthonormal Normalisation in Training Neural Networks
Generalisation of a deep neural network (DNN) is one major concern when ...
research
∙
02/24/2019
Rapidly Adapting Moment Estimation
Adaptive gradient methods such as Adam have been shown to be very effect...
research
∙
10/22/2018
DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score
We propose a training method for deep neural network (DNN)-based source ...
research
∙
02/24/2017