`

Timezone: »

 
Poster
Learning Domain Invariant Representations in Goal-conditioned Block MDPs
Beining Han · Chongyi Zheng · Harris Chan · Keiran Paster · Michael Zhang · Jimmy Ba

Thu Dec 09 04:30 PM -- 06:00 PM (PST) @

Deep Reinforcement Learning (RL) is successful in solving many complex Markov Decision Processes (MDPs) problems. However, agents often face unanticipated environmental changes after deployment in the real world. These changes are often spurious and unrelated to the underlying problem, such as background shifts for visual input agents. Unfortunately, deep RL policies are usually sensitive to these changes and fail to act robustly against them. This resembles the problem of domain generalization in supervised learning. In this work, we study this problem for goal-conditioned RL agents. We propose a theoretical framework in the Block MDP setting that characterizes the generalizability of goal-conditioned policies to new environments. Under this framework, we develop a practical method PA-SkewFit that enhances domain generalization. The empirical evaluation shows that our goal-conditioned RL agent can perform well in various unseen test environments, improving by 50\% over baselines.

Author Information

Beining Han (Tsinghua University)
Chongyi Zheng (CMU, Carnegie Mellon University)
Harris Chan (University of Toronto, Vector Institute)
Keiran Paster (University of Toronto)
Michael Zhang (University of Toronto / Vector Institute)

PhD student at the University of Toronto

Jimmy Ba (University of Toronto / Vector Institute)

More from the Same Authors

  • 2021 : BLAST: Latent Dynamics Models from Bootstrapping »
    Keiran Paster · Lev McKinney · Sheila McIlraith · Jimmy Ba
  • 2021 Poster: Clockwork Variational Autoencoders »
    Vaibhav Saxena · Jimmy Ba · Danijar Hafner
  • 2021 Poster: On the Estimation Bias in Double Q-Learning »
    Zhizhou Ren · Guangxiang Zhu · Hao Hu · Beining Han · Jianglun Chen · Chongjie Zhang
  • 2021 Poster: How does a Neural Network's Architecture Impact its Robustness to Noisy Labels? »
    Jingling Li · Mozhi Zhang · Keyulu Xu · John Dickerson · Jimmy Ba
  • 2021 Poster: Towards Understanding Cooperative Multi-Agent Q-Learning with Value Factorization »
    Jianhao Wang · Zhizhou Ren · Beining Han · Jianing Ye · Chongjie Zhang
  • 2020 : Contributed Talk #2: Evaluating Agents Without Rewards »
    Brendon Matusch · Danijar Hafner · Jimmy Ba
  • 2020 : Contributed Talk: Planning from Pixels using Inverse Dynamics Models »
    Keiran Paster · Sheila McIlraith · Jimmy Ba
  • 2020 Session: Orals & Spotlights Track 34: Deep Learning »
    Tuo Zhao · Jimmy Ba
  • 2019 : Posters »
    Colin Graber · Yuan-Ting Hu · Tiantian Fang · Jessica Hamrick · Giorgio Giannone · John Co-Reyes · Boyang Deng · Eric Crawford · Andrea Dittadi · Peter Karkus · Matthew Dirks · Rakshit Trivedi · Sunny Raj · Javier Felip Leon · Harris Chan · Jan Chorowski · Jeff Orchard · Aleksandar Stanić · Adam Kortylewski · Ben Zinberg · Chenghui Zhou · Wei Sun · Vikash Mansinghka · Chun-Liang Li · Marco Cusumano-Towner
  • 2019 : Poster Session »
    Eduard Gorbunov · Alexandre d'Aspremont · Lingxiao Wang · Liwei Wang · Boris Ginsburg · Alessio Quaglino · Camille Castera · Saurabh Adya · Diego Granziol · Rudrajit Das · Raghu Bollapragada · Fabian Pedregosa · Martin Takac · Majid Jahani · Sai Praneeth Karimireddy · Hilal Asi · Balint Daroczy · Leonard Adolphs · Aditya Rawal · Nicolas Brandt · Minhan Li · Giuseppe Ughi · Orlando Romero · Ivan Skorokhodov · Damien Scieur · Kiwook Bae · Konstantin Mishchenko · Rohan Anil · Vatsal Sharan · Aditya Balu · Chao Chen · Zhewei Yao · Tolga Ergen · Paul Grigas · Chris Junchi Li · Jimmy Ba · Stephen J Roberts · Sharan Vaswani · Armin Eftekhari · Chhavi Sharma
  • 2019 Poster: Lookahead Optimizer: k steps forward, 1 step back »
    Michael Zhang · James Lucas · Jimmy Ba · Geoffrey E Hinton
  • 2019 Poster: Graph Normalizing Flows »
    Jenny Liu · Aviral Kumar · Jimmy Ba · Jamie Kiros · Kevin Swersky
  • 2018 : Poster Session 1 + Coffee »
    Tom Van de Wiele · Rui Zhao · J. Fernando Hernandez-Garcia · Fabio Pardo · Xian Yeow Lee · Xiaolin Andy Li · Marcin Andrychowicz · Jie Tang · Suraj Nair · Juhyeon Lee · Cédric Colas · S. M. Ali Eslami · Yen-Chen Wu · Stephen McAleer · Ryan Julian · Yang Xue · Matthia Sabatelli · Pranav Shyam · Alexandros Kalousis · Giovanni Montana · Emanuele Pesce · Felix Leibfried · Zhanpeng He · Chunxiao Liu · Yanjun Li · Yoshihide Sawada · Alexander Pashevich · Tejas Kulkarni · Keiran Paster · Luca Rigazio · Quan Vuong · Hyunggon Park · Minhae Kwon · Rivindu Weerasekera · Shamane Siriwardhanaa · Rui Wang · Ozsel Kilinc · Keith Ross · Yizhou Wang · Simon Schmitt · Thomas Anthony · Evan Cater · Forest Agostinelli · Tegg Sung · Shirou Maruyama · Alexander Shmakov · Devin Schwab · Mohammad Firouzi · Glen Berseth · Denis Osipychev · Jesse Farebrother · Jianlan Luo · William Agnew · Peter Vrancx · Jonathan Heek · Catalin Ionescu · Haiyan Yin · Megumi Miyashita · Nathan Jay · Noga H. Rotman · Sam Leroux · Shaileshh Bojja Venkatakrishnan · Henri Schmidt · Jack Terwilliger · Ishan Durugkar · Jonathan Sauder · David Kas · Arash Tavakoli · Alain-Sam Cohen · Philip Bontrager · Adam Lerer · Thomas Paine · Ahmed Khalifa · Ruben Rodriguez · Avi Singh · Yiming Zhang
  • 2018 Poster: Reversible Recurrent Neural Networks »
    Matthew MacKay · Paul Vicol · Jimmy Ba · Roger Grosse