We present the MASH simulator, a benchmarking tool for both reinforcement and imitation learning algorithms. It comprises a 3D simulator which allows for evaluation on tasks based on motions in a physical environment whose complexities go well beyond that of typical 2D mazes. The simulator is composed of a graphics engine, Ogre 3D, to generate images corresponding to an avatar's views, and of a physical engine, Bullet, to simulate the avatar's physical interactions with the environment when moving around. We have developed a series of tasks and environments within this framework. To each is associated a reward function, and for some of them a teacher can be queried for the optimal action. The framework has been designed so that new tasks and environments can be easily created. This simulator has been developed in C++ and the code is publicly available under the GPLv3 open-source license.

Probabilistic programming languages (PPLs) such as Church have recently gained interest due to their expressiveness and ease-of-use. These languages support models that can be arbitrary composed and abstracted, and close the gap between traditional programming and probabilistic modeling. However debugging machine learning is still quite hard; it is difficult to identify whether a wrong prediction is due to inaccurate modeling, bug in the data representation, or the wrong choice of the approximate inference algorithm. Corresponding to the IDEs for traditional programming languages, there is a significant need for tools for probabilistic programming that enable the user to quickly create, debug, and iterate on complex models and inference.

In this demonstration, we present a first step towards integrated developing, debugging and visualization for PPLs. The proposed IDE is based on the read-eval-print loop (REPL) to allow quick iterations on writing and debugging, and consists of the following elements: (1) Editor (read): Users define the model and inference in a declarative, math-like language (with operators such as argmax) using a syntax highlighted code editor. (2) Build Automation (eval): Our underlying compilation engine transforms this declarative definition into factor graphs, and replaces the operators with efficient and/or approximate versions (such as Gibbs sampling, variational inference, gradient-based approaches, etc.) to generate low-level inference or learning code. (3) Debugging/Visualization (print): Our tool presents the underlying factor graph as an interactive UI element that supports clicking, drag and drop, hover, etc. to explore the structure and the potentials of the model. We visualize the results of inference in a graphical manner that adapts to the type of the result (bar charts for simple distributions, shaded maps for matrix-like objects, circles/arrows for NLP data types, etc.). For further fine-grained debugging, we can also surface intermediate results from inference, for example, visualizing the messages in belief propagation for each edge in the factor graph.

The combination of concise definition of the model and the inference, underlying compilation to factor graphs, and efficient implementation of operators using approximate inference enables users to write a variety of models, explore a number of different inference algorithms, and tackle tasks from a range of domains using an easy-to-use, platform independent (browser-based) interface.

Quick interaction between a human teacher and a learning machine presents numerous benefits and challenges when working with web-scale data. The human teacher guides the machine towards accomplishing the task of interest. The learning machine leverages big data to find examples that maximize the training value of its interaction with the teacher. When the teacher is restricted to labeling examples selected by the machine, this problem is an instance of active learning. When the teacher can provide additional information to the machine (e.g., suggestions on what examples or predictive features should be used) as the learning task progresses, then the problem becomes one of interactive learning. To accommodate the two-way communication channel needed for efficient interactive learning, the teacher and the machine need an environment that supports an interaction language. The machine can access, process, and summarize more examples than the teacher can see in a lifetime. Based on the machine’s output, the teacher can revise the definition of the task or make it more precise. Both the teacher and the machine continuously learn and benefit from the interaction. We have built a platform (ICE) to (1) produce valuable and deployable models and (2) support research on both the machine learning and user interface challenges of the interactive learning problem. The platform relies on a dedicated, low-latency, distributed, in-memory architecture that allows us to construct web-scale learning machines with quick interaction speed.

Tactile sensing affords robots the opportunity to dexterously manipulate objects in-hand without the need of strong object models and planning. Our demonstration focuses on learning for tactile, in-hand manipulation by robots. We address learning problems related to the control of objects in-hand, as well as perception problems encountered by a robot exploring its environment with a tactile sensor. We demonstrate applications for three specific learning prob- lems: learning to detect slip for grasp stability, learning to reposition objects in-hand, and learning to identify objects and object properties through tactile exploration. We address the problem of learning to detect slip of grasped objects. We show that the robot can learn a detector for slip events which generalizes to novel objects. We leverage this slip detector to produce a feedback controller that can stabilize objects during grasping and manipulation. Our work compares a number of supervised learning approaches and feature representations in order to achieve reliable slip detection. Tactile sensors provide observations of high enough dimension to cause prob- lems for traditional reinforcement learning methods. As such, we introduce a novel reinforcement learning (RL) algorithm which learns transition functions embedded in a reproducing kernel Hilbert space (RKHS). The resulting policy search algorithm provides robust policy updates which can efficiently deal with high-dimensional sensory input. We demonstrate the method on the problem of repositioning a grasped object in the hand. Finally, we present a method for learning to classify objects through tactile exploration. The robot collects data from a number of objects through various exploratory motions. The robot learns a classifier for each object to be used dur- ing exploration of its environment to detect objects in cluttered environments. Here again we compare a number of learning methods and features present in the literature and synthesize a method to best work in human environments the robot is likely to encounter. Users will be able to interact with a robot hand by giving it objects to grasp and attempting to remove these objects from the robot. The hand will also perform some basic in-hand manipulation tasks such as rolling the object between the fingers and rotating the object about a fixed grasp point. Users will also be able to interact with a touch sensor capable of classifying objects as well as semantic events such as slipping from a stable contact location.

This demo features an interactive autonomous wheelchair robot, called the SmartWheeler, designed with the aim of increasing the autonomy and safety of individuals with severe mobility impairments. The robot platform consists of a commercial power wheelchair, modified to be controlled by an on-board computer. It is further equipped with laser range finders and a Kinect sensor used for sensing the environment and building a map. The wheelchair can be given high-level commands (e.g. setting a location goal on a map), or lower-level commands (e.g. move forward, turn left/right, etc.) via a touchscreen interface. The commands are then interpreted and executed autonomously and safely by the robot, by taking into account static and dynamic obstacles in the environment. Several of the robot's software components rely on machine learning methods. One of the most recent developments is a new algorithm for detecting, tracking and following one or multiple people in real time using planar laser scans acquired onboard the robot.

We demonstrate a polyphonic transcription system capable of recognizing live music from an acoustic piano and representing it in symbolic form. In addition, we show how the system can be used to resynthesize performances by famous pianists using new instruments. Our transcription system uses a new probabilistic model that reflects the process by which discrete musical events give rise to acoustic signals that are then superimposed to produce the observed data. As a result, the inference procedure for our model naturally resolves the source separation problem introduced by the the piano’s polyphony. In order to adapt to the properties of a new instrument or acoustic environment being transcribed, we learn recording-specific spectral profiles and temporal envelopes in an unsupervised fashion.