Timezone: »

 
Workshop
Object Representations for Learning and Reasoning
William Agnew · Rim Assouel · Michael Chang · Antonia Creswell · Eliza Kosoy · Aravind Rajeswaran · Sjoerd van Steenkiste

Fri Dec 11 08:00 AM -- 07:15 PM (PST) @ None
Event URL: https://orlrworkshop.github.io/ »

Recent advances in deep reinforcement learning and robotics have enabled agents to achieve superhuman performance on a variety of challenging games and learn complex manipulation tasks. While these results are very promising, several open problems remain. In order to function in real-world environments, learned policies must be both robust to input perturbations and be able to rapidly generalize or adapt to novel situations. Moreover, to collaborate and live with humans in these environments, the goals and actions of embodied agents must be interpretable and compatible with human representations of knowledge. Hence, it is natural to consider how humans so successfully perceive, learn, and plan to build agents that are equally successful at solving real world tasks.
There is much evidence to suggest that objects are a core level of abstraction at which humans perceive and understand the world [8]. Objects have the potential to provide a compact, casual, robust, and generalizable representation of the world. Recently, there have been many advancements in scene representation, allowing scenes to be represented by their constituent objects, rather than at the level of pixels. While these works have shown promising results, there is still a lack of agreement on how to best represent objects, how to learn object representations, and how best to leverage them in agent training.
In this workshop we seek to build a consensus on what object representations should be by engaging with researchers from developmental psychology and by defining concrete tasks and capabilities that agents building on top of such abstract representations of the world should succeed at. We will discuss how object representations may be learned through invited presenters with expertise both in unsupervised and supervised object representation learning methods. Finally, we will host conversations and research on new frontiers in object learning.

Fri 8:00 a.m. - 8:15 a.m.
Introduction
William Agnew
Fri 8:15 a.m. - 9:00 a.m.

Elizabeth Spelke is the Marshall L. Berkman Professor of Psychology at Harvard University and an investigator at the NSF-MIT Center for Brains, Minds and Machines. Her laboratory focuses on the sources of uniquely human cognitive capacities, including capacities for formal mathematics, for constructing and using symbols, and for developing comprehensive taxonomies of objects. She probes the sources of these capacities primarily through behavioral research on human infants and preschool children, focusing on the origins and development of their understanding of objects, actions, people, places, number, and geometry. In collaboration with computational cognitive scientists, she aims to test computational models of infants’ cognitive capacities. In collaboration with economists, she has begun to take her research from the laboratory to the field, where randomized controlled experiments can serve to evaluate interventions, guided by research in cognitive science, that seek to enhance young children’s learning.

Fri 9:02 a.m. - 9:04 a.m.
Learning Object-Centric Video Models by Contrasting Sets (Lightning) Video
Fri 9:04 a.m. - 9:06 a.m.
Structure-Regularized Attention for Deformable Object Representation (Lightning) Video
Fri 9:06 a.m. - 9:08 a.m.
Learning Long-term Visual Dynamics with Region Proposal Interaction Networks (Lightning) Video
Fri 9:08 a.m. - 9:10 a.m.
Self-Supervised Attention-Aware Reinforcement Learning (Lightning) Video
Fri 9:10 a.m. - 9:12 a.m.
Emergence of compositional abstractions in human collaborative assembly (Lightning) Video
Fri 9:12 a.m. - 9:14 a.m.
Semantic State Representation for Reinforcement Learning (Lightning) Video
Fri 9:14 a.m. - 9:16 a.m.
Odd-One-Out Representation Learning (Lightning) Video
Fri 9:16 a.m. - 9:18 a.m.
Word(s) and Object(s): Grounded Language Learning In Information Retrieval (Lightning) Video
Fri 9:20 a.m. - 9:22 a.m.
Discrete Predictive Representation for Long-horizon Planning (Lightning) Video
Fri 9:22 a.m. - 9:24 a.m.
Dynamic Regions Graph Neural Networks for Spatio-Temporal Reasoning (Lightning) Video
Fri 9:26 a.m. - 9:28 a.m.
Dexterous Robotic Grasping with Object-Centric Visual Affordances (Lightning) Video
Fri 9:28 a.m. - 9:30 a.m.
Understanding designed objects by program synthesis (Lightning) Video
Fri 9:29 a.m. - 9:31 a.m.
Learning Embeddings that Capture Spatial Semantics for Indoor Navigation (Lightning) Video
Fri 9:30 a.m. - 10:30 a.m.
Poster Session A in GatherTown (Poster Session)
Fri 10:30 a.m. - 11:45 a.m.

How can we obtain object representations in real world environments? How can object representations be applied in robotics? Join us for a panel discussion with Jessica Hamrick, Irina Higgins, Michelle Lee, Josh Tenenbaum, moderated by Klaus Greff.

Jessica Hamrick, Klaus Greff, Michelle A. Lee, Irina Higgins, Josh Tenenbaum
Fri 11:45 a.m. - 12:25 p.m.
Break in GatherTown (Break)
Fri 12:25 p.m. - 12:55 p.m.
Video

Jessica Hamrick is a Senior Research Scientist at DeepMind, where she studies how to build machines that can flexibly build and deploy models of the world. Her work combines insights from cognitive science with structured relational architectures, model-based deep reinforcement learning, and planning. Jessica received a Ph.D. in Psychology from UC Berkeley in 2017, and an M.Eng. and B.S. in Computer Science from MIT in 2012.

Jessica Hamrick
Fri 12:55 p.m. - 1:25 p.m.
Video

Irina Higgins is a research scientist at DeepMind, where she works in the Frontiers team. Her work aims to bring together insights from the fields of neuroscience and physics to advance general artificial intelligence through improved representation learning. Before joining DeepMind, Irina was a British Psychological Society Undergraduate Award winner for her achievements as an undergraduate student in Experimental Psychology at Westminster University, followed by a DPhil at the Oxford Centre for Computational Neuroscience and Artificial Intelligence, where she focused on understanding the computational principles underlying speech processing in the auditory brain. During her DPhil, Irina also worked on developing poker AI, applying machine learning in the finance sector, and working on speech recognition at Google Research.

Irina Higgins
Fri 1:25 p.m. - 1:55 p.m.
Video

Sungjin Ahn is an Assistant Professor of Computer Science at Rutgers University and directs the Rutgers Machine Learning (RUML) lab. He is also affiliated with Rutgers Center for Cognitive Science. His research focus is on how an AI-agent can learn the structure and representations of the world in an unsupervised and compositional way, with a particular interest in object-centric learning. His approach to achieving this is based on deep learning, Bayesian modeling, reinforcement learning, and inspiration from cognitive & neuroscience. He received Ph.D. at the University of California, Irvine with Max Welling and did a postdoc with Yoshua Bengio at Mila. Then, he joined Rutgers University in Fall 2018. He has co-organized ICML 2020 Workshop on Object-Oriented Learning and received the ICML best paper award in ICML 2012.

Sungjin Ahn
Fri 1:55 p.m. - 2:07 p.m.
Contributed Talk : A Symmetric and Object-Centric World Model for Stochastic Environments (Talk) Video
Fri 2:07 p.m. - 2:19 p.m.
Contributed Talk : OGRE: An Object-based Generalization for Reasoning Environment (Talk) Video
Fri 2:19 p.m. - 2:49 p.m.
Video

Wilka Carvalho is a PhD Candidate in Computer Science at the University of Michigan–Ann Arbor where he is advised by Honglak Lee, Satinder Singh, and Richard Lewis. His long-term research goal is to develop cognitive theories of learning that help us understand how humans infer, reason with, and exploit the rich structure present in realistic visual scenes to enable sophisticated behavioral policies. Towards this end, he is studying how object-centric representation learning and reinforcement learning can bring us closer to human-level artificial intelligence. He is supported by an NSF GRFP Fellowship and a UM Rackham Merit Fellowship.

Wilka Carvalho
Fri 2:49 p.m. - 3:20 p.m.
Break in GatherTown (Break)
Fri 3:20 p.m. - 3:50 p.m.
Video

Renée Baillargeon is an Alumni Distinguished Professor of Psychology at the University of Illinois Urbana-Champaign. Her research examines cognitive development in infancy and focuses primarily on causal reasoning. In particular, she explores how infants make sense of the events they observe, and what explanatory frameworks and learning mechanisms enable them to do so. In addition to this primary focus on causal reasoning, she is interested in a broad range of related issues including object perception, categorization, object individuation, number, and executive-function skills.

Fri 3:50 p.m. - 4:20 p.m.

Object Representations for Robot Manipulation

Reasoning about objects is a fundamental task in robot manipulation. Different representations can have important repercussions on the capabilities and generality of a manipulation system. In this talk I will discuss different ways we represent and reason about objects, ranging from explicit 3D models to raw point clouds.

Fri 4:20 p.m. - 4:32 p.m.
Contributed Talk : Disentangling 3D Prototypical Networks for Few-Shot Concept Learning (Talk) Video
Fri 4:32 p.m. - 4:44 p.m.
Contributed Talk : Deep Affordance Foresight: Planning for What Can Be Done Next (Talk) Video
Fri 4:44 p.m. - 4:56 p.m.
Contributed talk : Estimating Mass Distribution of Articulated Objects using Non-prehensile Manipulation (Talk) Video
Fri 4:56 p.m. - 6:10 p.m.

What should be in an object representation, what should an object representation be able to do, and how do we measure and compare them? Join us for a panel discussion with Wilka Carvalho, Judy Fan, Tejas Kulkarni, and Chris Xie, moderated by Rachit Dubey.

, Wilka Carvalho, Judith Fan, Tejas Kulkarni, Christopher Xie
Fri 6:10 p.m. - 6:15 p.m.
Concluding Remarks
Fri 6:15 p.m. - 7:15 p.m.
Poster Session B in GatherTown (Poster Session)

Author Information

William Agnew (University of Washington)
Rim Assouel (umontreal.ca)
Michael Chang (University of California, Berkeley)

Ph.D. student at Berkeley AI Research, U.C. Berkeley B.S. in Computer Science from MIT Former research intern under Juergen Schmidhuber, Istituto Dalle Molle di Studi sull'Intelligenza Artificiale (IDSIA) Former undergraduate researcher under Joshua Tenenbaum and Antonio Torralba, MIT

Antonia Creswell (Deep Mind)
Eliza Kosoy (UC Berkeley)

Working with Alison Gopnik (and a few others at MIT and Berkeley) on the intersections of ai and child development

Aravind Rajeswaran (University of Washington)
Sjoerd van Steenkiste (Dalle Molle Institute for Artificial Intelligence Research (IDSIA))

More from the Same Authors