CYCLADES: Conflict-free Asynchronous Machine Learning

05/31/2016
by   Xinghao Pan, et al.
0

We present CYCLADES, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. CYCLADES is asynchronous during shared model updates, and requires no memory locking mechanisms, similar to HOGWILD!-type algorithms. Unlike HOGWILD!, CYCLADES introduces no conflicts during the parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent conflict-free nature and cache locality, our multi-core implementation of CYCLADES consistently outperforms HOGWILD!-type algorithms on sufficiently sparse datasets, leading to up to 40 implementation of SGD, and up to 5x gains over asynchronous implementations of variance reduction algorithms.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset