There has been recent very exciting advances in (deep) reinforcement learning, particularly in the areas of games and robotics. Yet perhaps the largest impact could come when reinforcement learning systems interact with people. In this tutorial we will discuss work on reinforcement learning for helping and assisting people, and frameworks and approaches for enabling people helping reinforcement learning. We will cover

Background on reinforcement learning. Reinforcement learning for people-focused applications Approaches for enabling people to assist reinforcement learners

A number of the ideas presented here will also be relevant to many high stakes reinforcement learning systems.

Target audience: The majority of the tutorial will be aimed at an audience who has a basic machine learning background (e.g. as acquired by a class or equivalent).

Learning objectives: Know some of the key technical challenges that arise for reinforcement learning in people-focusing domains; understand some of the algorithms and approaches that have been developed to address these challenges; become familiar with some of the other application areas that have also or can also benefit from reinforcement learning.

This tutorial will provide a gentle introduction into the foundations of statistical relational artificial intelligence, and will realize this by introducing the foundations of logic, of probability, of learning, and their respective combinations. Both predicate logic and probability theory extend propositional logic, one by adding relations, individuals and quantified variables, the other by allowing for measures over possible worlds and conditional queries. While logical and probabilistic approaches have often been studied and used independently within artificial intelligence, they are not in conflict with each other but they are synergistic. This explains why there has been a considerable body of research in combining first-order logic and probability over the last 25 years, evolving into what has come to be called Statistical Relational Artificial Intelligence (StarAI).

Relational probabilistic models — we use this term in the broad sense, meaning any models that combine relations and probabilities — form the basis of StarAI, and can be seen as combinations of probability and predicate calculus that allow for individuals and relations as well as probabilities. In building on top of relational models, StarAI goes far beyond reasoning, optimization, learning and acting optimally in terms of a fixed number of features or variables, as it is typically studied in machine learning, constraint satisfaction, probabilistic reasoning, and other areas of AI. Since StarAI draws upon ideas developed within many different fields, however, it can also be quite challenging for newcomers to get started and our tutorial precisely aims to provide this background.

Recent successes in computer vision, natural language processing and other areas of artificial intelligence have been largely driven by methods for sophisticated pattern recognition — most prominently deep neural networks. But human intelligence is more than just pattern recognition. In particular, it depends on a suite of cognitive capacities for modeling the world: for making judgment calls in ambiguous situations, explaining and understanding what we see, imagining things we could see but haven’t yet, solving problems and planning actions to make these things real, and building new models as we learn more about the world. We will talk about prospects for reverse-engineering these capacities at the heart of human intelligence, and using what we learn to make machines smarter in more human-like ways. We introduce basic concepts and techniques of probabilistic programs, inference programming and program induction, which together with tools from deep learning and modern video game engines provide an approach to capturing many aspects of everyday intelligence.

Specific units in our tutorial will show how:

(1) Defining probabilistic programs over algorithms and representations drawn from modern video game engines — graphics engines, physics engines, and planning engines — allows us to capture how people can perceive rich three-dimensional structure in visual scenes and objects, perceive and predict objects' motion based on their physical characteristics, and infer the mental states of other people from observing their actions.

(2) By formulating model learning as higher-order inference in these systems, we can construct ``program-learning programs’’. These programs can learn new concepts from just one or a few examples.

(3) It is possible to build probabilistic programming systems in which scalable, general-purpose, efficient inference and model discovery algorithms can be easily and flexibly programmed by end users. These languages provide powerful tools for robotics, interactive data analysis, and scientific discovery.