Timezone: »

Cyclades: Conflict-free Asynchronous Machine Learning
Xinghao Pan · Maximilian Lam · Stephen Tu · Dimitris Papailiopoulos · Ce Zhang · Michael Jordan · Kannan Ramchandran · Christopher Ré · Benjamin Recht

Wed Dec 07 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #98

We present Cyclades, a general framework for parallelizing stochastic optimization algorithms in a shared memory setting. Cyclades is asynchronous during model updates, and requires no memory locking mechanisms, similar to Hogwild!-type algorithms. Unlike Hogwild!, Cyclades introduces no conflicts during parallel execution, and offers a black-box analysis for provable speedups across a large family of algorithms. Due to its inherent cache locality and conflict-free nature, our multi-core implementation of Cyclades consistently outperforms Hogwild!-type algorithms on sufficiently sparse datasets, leading to up to 40% speedup gains compared to Hogwild!, and up to 5\times gains over asynchronous implementations of variance reduction algorithms.

Author Information

Xinghao Pan (UC Berkeley)
Maximilian Lam (UC Berkeley)
Stephen Tu (UC Berkeley)
Dimitris Papailiopoulos (University of Wisconsin-Madison)
Ce Zhang (Stanford)
Michael Jordan (UC Berkeley)
Kannan Ramchandran (UC Berkeley)
Christopher Ré (Stanford)
Benjamin Recht (UC Berkeley)

More from the Same Authors