Poster
Cyclades: Conflict-free Asynchronous Machine Learning
Xinghao Pan · Maximilian Lam · Stephen Tu · Dimitris Papailiopoulos · Ce Zhang · Michael Jordan · Kannan Ramchandran · Christopher Ré · Benjamin Recht

Wed Dec 7th 06:00 -- 09:30 PM @ Area 5+6+7+8 #98 #None
in Wednesday Posters »

We present Cyclades, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. Cyclades is asynchronous during model updates, and requires no memory locking mechanisms, similar to Hogwild!-type algorithms. Unlike Hogwild!, Cyclades introduces no conflicts during parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent cache locality and conflict-free nature, our multi-core implementation of Cyclades consistently outperforms Hogwild!-type algorithms on sufficiently sparse datasets, leading to up to 40% speedup gains compared to Hogwild!, and up to 5\times gains over asynchronous implementations of variance reduction algorithms.

Keywords: Large Scale Learning and Big Data · (Other) Optimization

Author Information

Xinghao Pan (UC Berkeley)
Maximilian Lam (UC Berkeley)
Stephen Tu (UC Berkeley)
Dimitrios Papailiopoulos (University of Wisconsin-Madison)
Ce Zhang (Stanford)
Michael Jordan (UC Berkeley)
Kannan Ramchandran (UC Berkeley)
Chris Ré (Stanford)
Benjamin Recht (UC Berkeley)

More from the Same Authors