Timezone: »

 
Poster
Communication Efficient Distributed Learning for Kernelized Contextual Bandits
Chuanhao Li · Huazheng Wang · Mengdi Wang · Hongning Wang

Thu Dec 01 02:00 PM -- 04:00 PM (PST) @ Hall J #307
in Poster Session 6 »
We tackle the communication efficiency challenge of learning kernelized contextual bandits in a distributed setting. Despite the recent advances in communication-efficient distributed bandit learning, existing solutions are restricted to simple models like multi-armed bandits and linear bandits, which hamper their practical utility. In this paper, instead of assuming the existence of a linear reward mapping from the features to the expected rewards, we consider non-linear reward mappings, by letting agents collaboratively search in a reproducing kernel Hilbert space (RKHS). This introduces significant challenges in communication efficiency as distributed kernel learning requires the transfer of raw data, leading to a communication cost that grows linearly w.r.t. time horizon $T$. We addresses this issue by equipping all agents to communicate via a common Nystr\"{o}m embedding that gets updated adaptively as more data points are collected. We rigorously proved that our algorithm can attain sub-linear rate in both regret and communication cost.
Keywords: contextual bandit · kernelized method · distributed learning · communication efficiency
[ Paper]  [ Slides]  [ Poster]  [ OpenReview] 

Author Information

Chuanhao Li (University of Virginia)
Huazheng Wang (Oregon State University)
Mengdi Wang (Princeton University)

Mengdi Wang is interested in data-driven stochastic optimization and applications in machine and reinforcement learning. She received her PhD in Electrical Engineering and Computer Science from Massachusetts Institute of Technology in 2013. At MIT, Mengdi was affiliated with the Laboratory for Information and Decision Systems and was advised by Dimitri P. Bertsekas. Mengdi became an assistant professor at Princeton in 2014. She received the Young Researcher Prize in Continuous Optimization of the Mathematical Optimization Society in 2016 (awarded once every three years).

Hongning Wang (University of Virginia)

More from the Same Authors