Timezone: »

Online Linear Regression and Its Application to Model-Based Reinforcement Learning
Alexander L Strehl · Michael L Littman

Mon Dec 03 10:30 AM -- 10:40 AM (PST) @ None #None

We provide a provably efficient algorithm for learning Markov Decision Processes (MDPs) with continuous state and action spaces in the online setting. Specifically, we take a model-based approach and show that a special type of online linear regression allows us to learn MDPs with (possibly kernalized) linearly parameterized dynamics. This result builds on Kearns and Singh's work that provides a provably efficient algorithm for finite state MDPs. Our approach is not restricted to the linear setting, and is applicable to other classes of continuous MDPs.

Author Information

Alexander L Strehl (Yahoo! Research)
Michael L Littman (Rutgers University)

Michael L. Littman is professor and chair of the Department of Computer Science at Rutgers University and directs the Rutgers Laboratory for Real-Life Reinforcement Learning (RL3). His research in machine learning examines algorithms for decision making under uncertainty. Littman has earned multiple awards for teaching and his research has been recognized with three best-paper awards on the topics of meta-learning for computer crossword solving, complexity analysis of planning under uncertainty, and algorithms for efficient reinforcement learning. He has served on the editorial boards for several machine-learning journals and was Programme Co-chair of ICML 2009.

More from the Same Authors