Timezone: »

 
Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning
Matthias Gerstgrasser · Tom Danino · Sarah Keren

@
in Deep Reinforcement Learning Workshop »
Event URL: https://openreview.net/forum?id=pkYYJaKevCN »

We present a novel multi-agent RL approach, Selective Multi-Agent PER, in which agents share with other agents a limited number of transitions they observe during training. They follow a similar heuristic as is used in (single-agent) Prioritized Experience Replay, and choose those transitions based on their td-error. The intuition behind this is that even a small number of relevant experiences from other agents could help each agent learn. Unlike many other multi-agent RL algorithms, this approach allows for largely decentralized training, requiring only a limited communication channel between agents. We show that our approach outperforms baseline no-sharing decentralized training. Further, sharing only a small number of experiences outperforms sharing all experiences between agents, and the performance uplift from selective experience sharing is robust across a range of hyperparameters.

[ Poster]  [ OpenReview]  [ Topia] 

Author Information

Matthias Gerstgrasser (Harvard University)
Tom Danino (Technion - Israel Institute of Technology, Technion)
Sarah Keren (Technion, Technion)

More from the Same Authors