Timezone: »

 
Poster
Countering Feedback Delays in Multi-Agent Learning
Zhengyuan Zhou · Panayotis Mertikopoulos · Nicholas Bambos · Peter W Glynn · Claire Tomlin

Wed Dec 06 06:30 PM -- 10:30 PM (PST) @ Pacific Ballroom #211
in Posters Wed »
We consider a model of game-theoretic learning based on \ac{OMD} with asynchronous and delayed feedback information. Instead of focusing on specific games, we consider a broad class of continuous games defined by the general equilibrium stability notion, which we call \emph{$\lambda$-variational stability}. Our first contribution is that, in this class of games, the actual sequence of play induced by \ac{OMD}-based learning converges to Nash equilibria provided that the feedback delays faced by the players are synchronous and bounded. Subsequently, to tackle fully decentralized, asynchronous environments with (possibly) unbounded delays between actions and feedback, we propose a variant of \ac{OMD} which we call \ac{DMD}, and which relies on the repeated leveraging of past information. With this modification, the algorithm converges to Nash equilibria with no feedback synchronicity assumptions and even when the delays grow superlinearly relative to the horizon of play.
Keywords: Learning Theory · Game Theory and Computational Economics
[ Paper] 

Author Information

Zhengyuan Zhou (Stanford University)
Panayotis Mertikopoulos (CNRS (French National Center for Scientific Research))
Nicholas Bambos (Stanford University)
Peter W Glynn (Stanford University)

Peter W. Glynn is the Thomas Ford Professor in the Department of Management Science and Engineering (MS&E) at Stanford University, and also holds a courtesy appointment in the Department of Electrical Engineering. He received his Ph.D in Operations Research from Stanford University in 1982. He then joined the faculty of the University of Wisconsin at Madison, where he held a joint appointment between the Industrial Engineering Department and Mathematics Research Center, and courtesy appointments in Computer Science and Mathematics. In 1987, he returned to Stanford, where he joined the Department of Operations Research. He was Director of Stanford's Institute for Computational and Mathematical Engineering from 2006 until 2010 and served as Chair of MS&E from 2011 through 2015. He is a Fellow of INFORMS and a Fellow of the Institute of Mathematical Statistics, and was an IMS Medallion Lecturer in 1995 and INFORMS Markov Lecturer in 2014. He was co-winner of the Outstanding Publication Awards from the INFORMS Simulation Society in 1993, 2008, and 2016, was a co-winner of the Best (Biannual) Publication Award from the INFORMS Applied Probability Society in 2009, and was the co-winner of the John von Neumann Theory Prize from INFORMS in 2010. In 2012, he was elected to the National Academy of Engineering. He was Founding Editor-in-Chief of Stochastic Systems and is currently Editor-in-Chief of Journal of Applied Probability and Advances in Applied Probability. His research interests lie in simulation, computational probability, queueing theory, statistical inference for stochastic processes, and stochastic modeling.

Claire Tomlin (UC Berkeley)

More from the Same Authors