Asymptotic Network Independence in Distributed Optimization for Machine Learning

06/28/2019
by   Alex Olshevsky, et al.
0

We provide a discussion of several recent results which have overcome a key barrier in distributed optimization for machine learning. Our focus is the so-called network independence property, which is achieved whenever a distributed method executed over a network of n nodes achieves comparable performance to a centralized method with the same computational power as the entire network. We explain this property through an example involving of training ML models and sketch a short mathematical analysis.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset