Learning and Planning from Batch Time Series Data
Daniel Lizotte · Michael Bowling · Susan Murphy · Joelle Pineau · Sandeep Vijan

Sat Dec 11th 07:30 AM -- 06:30 PM @ Hilton: Sutcliffe A
Event URL: »

Intended Audience: Researchers interested in models and algorithms for learning and planning from batches of time series, including those interested in batch reinforcement learning, dynamic Bayes nets, dynamical systems, and similar topics. Also, researchers interested in any applications where such algorithms and models can be of use, for example in medicine and robotics.

Overview: Consider the problem of learning a model or control policy from a batch of trajectories collected a priori that record observations over time. This scenario presents an array of practical challenges. For example, batch data are often noisy and/or partially missing. The data may be high-dimensional because the data collector may not know a priori which observations are useful for decision making. In fact, a data collector may not even have a clear idea of which observations should be used to measure the quality of a policy. Finally, even given low-noise data with a few useful state features and a well-defined objective, the performance of the learner can only be evaluated using the same batch of data that was available for learning.

The above challenges encountered in batch learning and planning from time series data are beginning to be addressed by adapting techniques that have proven useful in regression and classification. Careful modelling, filtering, or smoothing could mitigate noisy or missing observations. Appropriate regularization could be used for feature selection. Methods from multi-criterion optimization could be useful for choosing a performance measure. Specialized data re-sampling methods could yield valid assessments of policy performance when gathering new on-policy data is not possible.

As applications of reinforcement learning and related methods have become more widespread, practitioners have encountered the above challenges along with many others, and they have begun to develop and adapt a variety of methods from other areas of machine learning and statistics to address these challenges. The goal of our workshop is to further this development by bringing together researchers who are interested in learning and planning methods for batch time series data and researchers who are interested in applying these methods in medicine, robotics, and other relevant domains. Longer term we hope to jump-start synergistic collaborations aimed at improving the quality of learning and planning from training sets of time series for use in medical applications.

Author Information

Dan Lizotte (The University of Western Ontario)
Michael Bowling (DeepMind / University of Alberta)
Susan Murphy (University of Michigan)
Joelle Pineau (McGill University)

Joelle Pineau is an Associate Professor and William Dawson Scholar at McGill University where she co-directs the Reasoning and Learning Lab. She also leads the Facebook AI Research lab in Montreal, Canada. She holds a BASc in Engineering from the University of Waterloo, and an MSc and PhD in Robotics from Carnegie Mellon University. Dr. Pineau's research focuses on developing new models and algorithms for planning and learning in complex partially-observable domains. She also works on applying these algorithms to complex problems in robotics, health care, games and conversational agents. She serves on the editorial board of the Journal of Artificial Intelligence Research and the Journal of Machine Learning Research and is currently President of the International Machine Learning Society. She is a recipient of NSERC's E.W.R. Steacie Memorial Fellowship (2018), a Fellow of the Association for the Advancement of Artificial Intelligence (AAAI), a Senior Fellow of the Canadian Institute for Advanced Research (CIFAR) and in 2016 was named a member of the College of New Scholars, Artists and Scientists by the Royal Society of Canada.

Sandeep Vijan (University of Michigan)

More from the Same Authors