Workshop: Object Representations for Learning and Reasoning

William Agnew, Rim Assouel, Michael Chang, Antonia Creswell, Eliza Kosoy, Aravind Rajeswaran, Sjoerd van Steenkiste

2020-12-11T08:00:00-08:00 - 2020-12-11T19:15:00-08:00
Abstract: Recent advances in deep reinforcement learning and robotics have enabled agents to achieve superhuman performance on a variety of challenging games and learn complex manipulation tasks. While these results are very promising, several open problems remain. In order to function in real-world environments, learned policies must be both robust to input perturbations and be able to rapidly generalize or adapt to novel situations. Moreover, to collaborate and live with humans in these environments, the goals and actions of embodied agents must be interpretable and compatible with human representations of knowledge. Hence, it is natural to consider how humans so successfully perceive, learn, and plan to build agents that are equally successful at solving real world tasks.
There is much evidence to suggest that objects are a core level of abstraction at which humans perceive and understand the world [8]. Objects have the potential to provide a compact, casual, robust, and generalizable representation of the world. Recently, there have been many advancements in scene representation, allowing scenes to be represented by their constituent objects, rather than at the level of pixels. While these works have shown promising results, there is still a lack of agreement on how to best represent objects, how to learn object representations, and how best to leverage them in agent training.
In this workshop we seek to build a consensus on what object representations should be by engaging with researchers from developmental psychology and by defining concrete tasks and capabilities that agents building on top of such abstract representations of the world should succeed at. We will discuss how object representations may be learned through invited presenters with expertise both in unsupervised and supervised object representation learning methods. Finally, we will host conversations and research on new frontiers in object learning.

Video

Chat

Chat is not available.

Schedule

2020-12-11T08:00:00-08:00 - 2020-12-11T08:15:00-08:00
Introduction
William Agnew
2020-12-11T08:15:00-08:00 - 2020-12-11T09:00:00-08:00
Keynote: Elizabeth Spelke
Elizabeth Spelke is the Marshall L. Berkman Professor of Psychology at Harvard University and an investigator at the NSF-MIT Center for Brains, Minds and Machines. Her laboratory focuses on the sources of uniquely human cognitive capacities, including capacities for formal mathematics, for constructing and using symbols, and for developing comprehensive taxonomies of objects. She probes the sources of these capacities primarily through behavioral research on human infants and preschool children, focusing on the origins and development of their understanding of objects, actions, people, places, number, and geometry. In collaboration with computational cognitive scientists, she aims to test computational models of infants’ cognitive capacities. In collaboration with economists, she has begun to take her research from the laboratory to the field, where randomized controlled experiments can serve to evaluate interventions, guided by research in cognitive science, that seek to enhance young children’s learning.
2020-12-11T09:02:00-08:00 - 2020-12-11T09:04:00-08:00
Learning Object-Centric Video Models by Contrasting Sets
2020-12-11T09:04:00-08:00 - 2020-12-11T09:06:00-08:00
Structure-Regularized Attention for Deformable Object Representation
2020-12-11T09:06:00-08:00 - 2020-12-11T09:08:00-08:00
Learning Long-term Visual Dynamics with Region Proposal Interaction Networks
2020-12-11T09:08:00-08:00 - 2020-12-11T09:10:00-08:00
Self-Supervised Attention-Aware Reinforcement Learning
2020-12-11T09:10:00-08:00 - 2020-12-11T09:12:00-08:00
Emergence of compositional abstractions in human collaborative assembly
2020-12-11T09:12:00-08:00 - 2020-12-11T09:14:00-08:00
Semantic State Representation for Reinforcement Learning
2020-12-11T09:14:00-08:00 - 2020-12-11T09:16:00-08:00
Odd-One-Out Representation Learning
2020-12-11T09:16:00-08:00 - 2020-12-11T09:18:00-08:00
Word(s) and Object(s): Grounded Language Learning In Information Retrieval
2020-12-11T09:20:00-08:00 - 2020-12-11T09:22:00-08:00
Discrete Predictive Representation for Long-horizon Planning
2020-12-11T09:22:00-08:00 - 2020-12-11T09:24:00-08:00
Dynamic Regions Graph Neural Networks for Spatio-Temporal Reasoning
2020-12-11T09:26:00-08:00 - 2020-12-11T09:28:00-08:00
Dexterous Robotic Grasping with Object-Centric Visual Affordances
2020-12-11T09:28:00-08:00 - 2020-12-11T09:30:00-08:00
Understanding designed objects by program synthesis
2020-12-11T09:29:00-08:00 - 2020-12-11T09:31:00-08:00
Learning Embeddings that Capture Spatial Semantics for Indoor Navigation
2020-12-11T09:30:00-08:00 - 2020-12-11T10:30:00-08:00
Poster Session A in GatherTown
2020-12-11T10:30:00-08:00 - 2020-12-11T11:45:00-08:00
Panel Discussion
Jessica Hamrick, Klaus Greff, Michelle A. Lee, Irina Higgins, Josh Tenenbaum
How can we obtain object representations in real world environments? How can object representations be applied in robotics? Join us for a panel discussion with Jessica Hamrick, Irina Higgins, Michelle Lee, Josh Tenenbaum, moderated by Klaus Greff.
2020-12-11T11:45:00-08:00 - 2020-12-11T12:25:00-08:00
Break in GatherTown
2020-12-11T12:25:00-08:00 - 2020-12-11T12:55:00-08:00
Invited Talk: Jessica Hamrick
Jessica Hamrick
Jessica Hamrick is a Senior Research Scientist at DeepMind, where she studies how to build machines that can flexibly build and deploy models of the world. Her work combines insights from cognitive science with structured relational architectures, model-based deep reinforcement learning, and planning. Jessica received a Ph.D. in Psychology from UC Berkeley in 2017, and an M.Eng. and B.S. in Computer Science from MIT in 2012.
2020-12-11T12:55:00-08:00 - 2020-12-11T13:25:00-08:00
Invited Talk: Irina Higgins
Irina Higgins
Irina Higgins is a research scientist at DeepMind, where she works in the Frontiers team. Her work aims to bring together insights from the fields of neuroscience and physics to advance general artificial intelligence through improved representation learning. Before joining DeepMind, Irina was a British Psychological Society Undergraduate Award winner for her achievements as an undergraduate student in Experimental Psychology at Westminster University, followed by a DPhil at the Oxford Centre for Computational Neuroscience and Artificial Intelligence, where she focused on understanding the computational principles underlying speech processing in the auditory brain. During her DPhil, Irina also worked on developing poker AI, applying machine learning in the finance sector, and working on speech recognition at Google Research.
2020-12-11T13:25:00-08:00 - 2020-12-11T13:55:00-08:00
Invited Talk: Sungjin Ahn
Sungjin Ahn
Sungjin Ahn is an Assistant Professor of Computer Science at Rutgers University and directs the Rutgers Machine Learning (RUML) lab. He is also affiliated with Rutgers Center for Cognitive Science. His research focus is on how an AI-agent can learn the structure and representations of the world in an unsupervised and compositional way, with a particular interest in object-centric learning. His approach to achieving this is based on deep learning, Bayesian modeling, reinforcement learning, and inspiration from cognitive & neuroscience. He received Ph.D. at the University of California, Irvine with Max Welling and did a postdoc with Yoshua Bengio at Mila. Then, he joined Rutgers University in Fall 2018. He has co-organized ICML 2020 Workshop on Object-Oriented Learning and received the ICML best paper award in ICML 2012.
2020-12-11T13:55:00-08:00 - 2020-12-11T14:07:00-08:00
Contributed Talk : A Symmetric and Object-Centric World Model for Stochastic Environments
2020-12-11T14:07:00-08:00 - 2020-12-11T14:19:00-08:00
Contributed Talk : OGRE: An Object-based Generalization for Reasoning Environment
2020-12-11T14:19:00-08:00 - 2020-12-11T14:49:00-08:00
Invited Talk: Wilka Carvalho
Wilka Carvalho
Wilka Carvalho is a PhD Candidate in Computer Science at the University of Michigan–Ann Arbor where he is advised by Honglak Lee, Satinder Singh, and Richard Lewis. His long-term research goal is to develop cognitive theories of learning that help us understand how humans infer, reason with, and exploit the rich structure present in realistic visual scenes to enable sophisticated behavioral policies. Towards this end, he is studying how object-centric representation learning and reinforcement learning can bring us closer to human-level artificial intelligence. He is supported by an NSF GRFP Fellowship and a UM Rackham Merit Fellowship.
2020-12-11T14:49:00-08:00 - 2020-12-11T15:20:00-08:00
Break in GatherTown
2020-12-11T15:20:00-08:00 - 2020-12-11T15:50:00-08:00
Invited Talk: Renée Baillargeon
Renée Baillargeon is an Alumni Distinguished Professor of Psychology at the University of Illinois Urbana-Champaign. Her research examines cognitive development in infancy and focuses primarily on causal reasoning. In particular, she explores how infants make sense of the events they observe, and what explanatory frameworks and learning mechanisms enable them to do so. In addition to this primary focus on causal reasoning, she is interested in a broad range of related issues including object perception, categorization, object individuation, number, and executive-function skills.
2020-12-11T15:50:00-08:00 - 2020-12-12T16:20:00-08:00
Invited Talk: Dieter Fox
Object Representations for Robot Manipulation Reasoning about objects is a fundamental task in robot manipulation. Different representations can have important repercussions on the capabilities and generality of a manipulation system. In this talk I will discuss different ways we represent and reason about objects, ranging from explicit 3D models to raw point clouds.
2020-12-11T16:20:00-08:00 - 2020-12-11T16:32:00-08:00
Contributed Talk : Disentangling 3D Prototypical Networks for Few-Shot Concept Learning
2020-12-11T16:32:00-08:00 - 2020-12-11T16:44:00-08:00
Contributed Talk : Deep Affordance Foresight: Planning for What Can Be Done Next
2020-12-11T16:44:00-08:00 - 2020-12-11T16:56:00-08:00
Contributed talk : Estimating Mass Distribution of Articulated Objects using Non-prehensile Manipulation
2020-12-11T16:56:00-08:00 - 2020-12-11T18:10:00-08:00
Panel
, Wilka Carvalho, Judith Fan, Tejas Kulkarni, Christopher Xie
What should be in an object representation, what should an object representation be able to do, and how do we measure and compare them? Join us for a panel discussion with Wilka Carvalho, Judy Fan, Tejas Kulkarni, and Chris Xie, moderated by Rachit Dubey.
2020-12-11T18:10:00-08:00 - 2020-12-11T18:15:00-08:00
Concluding Remarks
2020-12-11T18:15:00-08:00 - 2020-12-11T19:15:00-08:00
Poster Session B in GatherTown