Timezone: »

 
Poster
Value Propagation for Decentralized Networked Deep Multi-agent Reinforcement Learning
Chao Qu · Shie Mannor · Huan Xu · Yuan Qi · Le Song · Junwu Xiong

Tue Dec 10 05:30 PM -- 07:30 PM (PST) @ East Exhibition Hall B + C #200
in Reinforcement Learning and Planning -- Multi-Agent RL »
We consider the networked multi-agent reinforcement learning (MARL) problem in a fully decentralized setting, where agents learn to coordinate to achieve joint success. This problem is widely encountered in many areas including traffic control, distributed control, and smart grids. We assume each agent is located at a node of a communication network and can exchange information only with its neighbors. Using softmax temporal consistency, we derive a primal-dual decentralized optimization method and obtain a principled and data-efficient iterative algorithm named {\em value propagation}. We prove a non-asymptotic convergence rate of $\mathcal{O}(1/T)$ with nonlinear function approximation. To the best of our knowledge, it is the first MARL algorithm with a convergence guarantee in the control, off-policy, non-linear function approximation, fully decentralized setting.
Keywords: Multi-Agent RL · Reinforcement Learning · Reinforcement Learning and Planning
[ Paper]  [ Slides] 

Author Information

Chao Qu (Ant Financial Services Group)
Shie Mannor (Technion)
Huan Xu (Georgia Inst. of Technology)
Yuan Qi (Ant Financial Services Group)
Le Song (Ant Financial Services Group)
Junwu Xiong (Ant Financial Services Group)

More from the Same Authors