Timezone: »

Cyclades: Conflict-free Asynchronous Machine Learning
Xinghao Pan · Maximilian Lam · Stephen Tu · Dimitris Papailiopoulos · Ce Zhang · Michael Jordan · Kannan Ramchandran · Christopher Ré · Benjamin Recht

Wed Dec 09:00 AM -- 12:30 PM PST @ Area 5+6+7+8 #98 #None

We present Cyclades, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. Cyclades is asynchronous during model updates, and requires no memory locking mechanisms, similar to Hogwild!-type algorithms. Unlike Hogwild!, Cyclades introduces no conflicts during parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent cache locality and conflict-free nature, our multi-core implementation of Cyclades consistently outperforms Hogwild!-type algorithms on sufficiently sparse datasets, leading to up to 40% speedup gains compared to Hogwild!, and up to 5\times gains over asynchronous implementations of variance reduction algorithms.

Author Information

Xinghao Pan (UC Berkeley)
Maximilian Lam (UC Berkeley)
Stephen Tu (UC Berkeley)
Dimitrios Papailiopoulos (University of Wisconsin-Madison)
Ce Zhang (Stanford)
Michael Jordan (UC Berkeley)
Kannan Ramchandran (UC Berkeley)
Chris Ré (Stanford)
Benjamin Recht (UC Berkeley)

More from the Same Authors