Timezone: »
Poster
Regression Planning Networks
Danfei Xu · Roberto Martín-Martín · De-An Huang · Yuke Zhu · Silvio Savarese · Li Fei-Fei
Thu Dec 12 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #219
Recent learning-to-plan methods have shown promising results on planning directly from observation space. Yet, their ability to plan for long-horizon tasks is limited by the accuracy of the prediction model. On the other hand, classical symbolic planners show remarkable capabilities in solving long-horizon tasks, but they require predefined symbolic rules and symbolic states, restricting their real-world applicability. In this work, we combine the benefits of these two paradigms and propose a learning-to-plan method that can directly generate a long-term symbolic plan conditioned on high-dimensional observations. We borrow the idea of regression (backward) planning from classical planning literature and introduce Regression Planning Networks (RPN), a neural network architecture that plans backward starting at a task goal and generates a sequence of intermediate goals that reaches the current observation. We show that our model not only inherits many favorable traits from symbolic planning --including the ability to solve previously unseen tasks-- but also can learn from visual inputs in an end-to-end manner. We evaluate the capabilities of RPN in a grid world environment and a simulated 3D kitchen environment featuring complex visual scenes and long task horizon, and show that it achieves near-optimal performance in completely new task instances.
Author Information
Danfei Xu (Stanford University)
Roberto Martín-Martín (Stanford University)
De-An Huang (Stanford University)
Yuke Zhu (University of Texas - Austin)
Silvio Savarese (Stanford University)
Li Fei-Fei (Stanford University)
More from the Same Authors
-
2021 : Task-Independent Causal State Abstraction »
Zizhao Wang · Xuesu Xiao · Yuke Zhu · Peter Stone -
2021 : Reinforcement Learning in Factored Action Spaces using Tensor Decompositions »
Anuj Mahajan · Mikayel Samvelyan · Lei Mao · Viktor Makoviichuk · Animesh Garg · Jean Kossaifi · Shimon Whiteson · Yuke Zhu · Anima Anandkumar -
2021 : Neural Abstructions: Abstractions that Support Construction for Grounded Language Learning »
Kaylee Burns · Christopher D Manning · Li Fei-Fei -
2021 : What Matters in Learning from Offline Human Demonstrations for Robot Manipulation »
Ajay Mandlekar · Danfei Xu · Josiah Wong · Chen Wang · Li Fei-Fei · Silvio Savarese · Yuke Zhu · Roberto Martín-Martín -
2022 : VIMA: General Robot Manipulation with Multimodal Prompts »
Yunfan Jiang · Agrim Gupta · Zichen Zhang · Guanzhi Wang · Yongqiang Dou · Yanjun Chen · Fei-Fei Li · Anima Anandkumar · Yuke Zhu · Linxi Fan -
2022 : Robust Trajectory Prediction against Adversarial Attacks »
Yulong Cao · Danfei Xu · Xinshuo Weng · Zhuoqing Morley Mao · Anima Anandkumar · Chaowei Xiao · Marco Pavone -
2022 : AdvDO: Realistic Adversarial Attacks for Trajectory Prediction »
Yulong Cao · Chaowei Xiao · Anima Anandkumar · Danfei Xu · Marco Pavone -
2022 : Raisin: Residual Algorithms for Versatile Offline Reinforcement Learning »
Braham Snyder · Yuke Zhu -
2023 Poster: UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild »
Can Qin · Shu Zhang · Ning Yu · Yihao Feng · Xinyi Yang · Yingbo Zhou · Huan Wang · Juan Carlos Niebles · Caiming Xiong · Silvio Savarese · Stefano Ermon · Yun Fu · Ran Xu -
2023 Poster: Cross-Episodic Curriculum for Transformer Agents »
Lucy Xiaoyang Shi · Yunfan Jiang · Jake Grigsby · Linxi Fan · Yuke Zhu -
2023 Poster: LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning »
Bo Liu · Yifeng Zhu · Chongkai Gao · Yihao Feng · Qiang Liu · Yuke Zhu · Peter Stone -
2022 Poster: Pre-Trained Language Models for Interactive Decision-Making »
Shuang Li · Xavier Puig · Chris Paxton · Yilun Du · Clinton Wang · Linxi Fan · Tao Chen · De-An Huang · Ekin Akyürek · Anima Anandkumar · Jacob Andreas · Igor Mordatch · Antonio Torralba · Yuke Zhu -
2022 Poster: MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge »
Linxi Fan · Guanzhi Wang · Yunfan Jiang · Ajay Mandlekar · Yuncong Yang · Haoyi Zhu · Andrew Tang · De-An Huang · Yuke Zhu · Anima Anandkumar -
2022 Poster: CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning »
Hung Le · Yue Wang · Akhilesh Deepak Gotmare · Silvio Savarese · Steven Chu Hong Hoi -
2021 : Reinforcement Learning in Factored Action Spaces using Tensor Decompositions »
Anuj Mahajan · Mikayel Samvelyan · Lei Mao · Viktor Makoviichuk · Animesh Garg · Jean Kossaifi · Shimon Whiteson · Yuke Zhu · Anima Anandkumar -
2020 : Closing remarks from Fei-Fei Li, Sequoia Professor of Computer Science, Stanford University & Co-Director of Stanford’s Human-Centered AI Institute »
Li Fei-Fei -
2020 : Q/A for invited talk #5 »
Li Fei-Fei -
2020 : Creating diverse tasks to catalyze robot learning »
Li Fei-Fei -
2020 Poster: Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning »
Weili Nie · Zhiding Yu · Lei Mao · Ankit Patel · Yuke Zhu · Anima Anandkumar -
2020 Spotlight: Bongard-LOGO: A New Benchmark for Human-Level Concept Learning and Reasoning »
Weili Nie · Zhiding Yu · Lei Mao · Ankit Patel · Yuke Zhu · Anima Anandkumar -
2019 Poster: Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks »
Vineet Kosaraju · Amir Sadeghian · Roberto Martín-Martín · Ian Reid · Hamid Rezatofighi · Silvio Savarese -
2019 Poster: HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models »
Sharon Zhou · Mitchell Gordon · Ranjay Krishna · Austin Narcomey · Li Fei-Fei · Michael Bernstein -
2019 Oral: HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models »
Sharon Zhou · Mitchell Gordon · Ranjay Krishna · Austin Narcomey · Li Fei-Fei · Michael Bernstein -
2018 Workshop: NIPS Workshop on Machine Learning for Intelligent Transportation Systems 2018 »
Li Erran Li · Anca Dragan · Juan Carlos Niebles · Silvio Savarese -
2018 Poster: Learning to Play With Intrinsically-Motivated, Self-Aware Agents »
Nick Haber · Damian Mrowca · Stephanie Wang · Li Fei-Fei · Daniel Yamins -
2018 Poster: Learning to Decompose and Disentangle Representations for Video Prediction »
Jun-Ting Hsieh · Bingbin Liu · De-An Huang · Li Fei-Fei · Juan Carlos Niebles -
2018 Poster: Generalizing to Unseen Domains via Adversarial Data Augmentation »
Riccardo Volpi · Hongseok Namkoong · Ozan Sener · John Duchi · Vittorio Murino · Silvio Savarese -
2018 Poster: Flexible neural representation for physics prediction »
Damian Mrowca · Chengxu Zhuang · Elias Wang · Nick Haber · Li Fei-Fei · Josh Tenenbaum · Daniel Yamins -
2017 Workshop: 2017 NIPS Workshop on Machine Learning for Intelligent Transportation Systems »
Li Erran Li · Anca Dragan · Juan Carlos Niebles · Silvio Savarese -
2017 : Keynote II: Fei-Fei Li, Stanford »
Li Fei-Fei -
2017 Poster: Label Efficient Learning of Transferable Representations acrosss Domains and Tasks »
Zelun Luo · Yuliang Zou · Judy Hoffman · Li Fei-Fei -
2016 : Knowledge Acquisition for Visual Question Answering via Iterative Querying »
Yuke Zhu · Joseph Lim · Li Fei-Fei -
2014 Poster: Deep Fragment Embeddings for Bidirectional Image Sentence Mapping »
Andrej Karpathy · Armand Joulin · Li Fei-Fei -
2012 Workshop: Big Data Meets Computer Vision: First International Workshop on Large Scale Visual Recognition and Retrieval »
Jia Deng · Samy Bengio · Yuanqing Lin · Li Fei-Fei -
2012 Poster: Shifting Weights: Adapting Object Detectors from Image to Video »
Kevin Tang · Vignesh Ramanathan · Li Fei-Fei · Daphne Koller -
2012 Demonstration: EVA: Engine for Visual Annotation »
Jia Deng · Joanathan Krause · Zhiheng Huang · Alexander C Berg · Li Fei-Fei -
2011 Poster: Fast and Balanced: Efficient Label Tree Learning for Large Scale Object Recognition »
Jia Deng · Sanjeev Satheesh · Alexander C Berg · Li Fei-Fei -
2011 Poster: Large-Scale Category Structure Aware Image Categorization »
Bin Zhao · Li Fei-Fei · Eric Xing -
2010 Session: Oral Session 10 »
Li Fei-Fei -
2010 Poster: Large Margin Learning of Upstream Scene Understanding Models »
Jun Zhu · Li-Jia Li · Li Fei-Fei · Eric Xing -
2010 Poster: Object Bank: A High-Level Image Representation for Scene Classification & Semantic Feature Sparsification »
Li-Jia Li · Hao Su · Eric Xing · Li Fei-Fei