Timezone: »

 
Spotlight
Learning to Play No-Press Diplomacy with Best Response Policy Iteration
Thomas Anthony · Tom Eccles · Andrea Tacchetti · János Kramár · Ian Gemp · Thomas Hudson · Nicolas Porcel · Marc Lanctot · Julien Perolat · Richard Everett · Satinder Singh · Thore Graepel · Yoram Bachrach

Thu Dec 10 08:20 AM -- 08:30 AM (PST) @ Orals & Spotlights: Reinforcement Learning
in Orals & Spotlights Track 31: Reinforcement Learning »

Recent advances in deep reinforcement learning (RL) have led to considerable progress in many 2-player zero-sum games, such as Go, Poker and Starcraft. The purely adversarial nature of such games allows for conceptually simple and principled application of RL methods. However real-world settings are many-agent, and agent interactions are complex mixtures of common-interest and competitive aspects. We consider Diplomacy, a 7-player board game designed to accentuate dilemmas resulting from many-agent interactions. It also features a large combinatorial action space and simultaneous moves, which are challenging for RL algorithms. We propose a simple yet effective approximate best response operator, designed to handle large combinatorial action spaces and simultaneous moves. We also introduce a family of policy iteration methods that approximate fictitious play. With these methods, we successfully apply RL to Diplomacy: we show that our agents convincingly outperform the previous state-of-the-art, and game theoretic equilibrium analysis shows that the new process yields consistent improvements.

Author Information

Thomas Anthony (DeepMind)
Tom Eccles (DeepMind)
Andrea Tacchetti (DeepMind)
János Kramár (DeepMind)
Ian Gemp (DeepMind)
Thomas Hudson (DeepMind)
Nicolas Porcel (DeepMind)
Marc Lanctot (DeepMind)
Julien Perolat (DeepMind)
Richard Everett (DeepMind)
Satinder Singh (DeepMind)
Thore Graepel (DeepMind)
Yoram Bachrach (Google DeepMind)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors