Timezone: »

Transfer of Deep Reactive Policies for MDP Planning
Aniket (Nick) Bajpai · Sankalp Garg · Mausam

Wed Dec 05 02:00 PM -- 04:00 PM (PST) @ Room 517 AB #118

Domain-independent probabilistic planners input an MDP description in a factored representation language such as PPDDL or RDDL, and exploit the specifics of the representation for faster planning. Traditional algorithms operate on each problem instance independently, and good methods for transferring experience from policies of other instances of a domain to a new instance do not exist. Recently, researchers have begun exploring the use of deep reactive policies, trained via deep reinforcement learning (RL), for MDP planning domains. One advantage of deep reactive policies is that they are more amenable to transfer learning.

In this paper, we present the first domain-independent transfer algorithm for MDP planning domains expressed in an RDDL representation. Our architecture exploits the symbolic state configuration and transition function of the domain (available via RDDL) to learn a shared embedding space for states and state-action pairs for all problem instances of a domain. We then learn an RL agent in the embedding space, making a near zero-shot transfer possible, i.e., without much training on the new instance, and without using the domain simulator at all. Experiments on three different benchmark domains underscore the value of our transfer algorithm. Compared against planning from scratch, and a state-of-the-art RL transfer algorithm, our transfer solution has significantly superior learning curves.

Author Information

Aniket (Nick) Bajpai (MIT)
Sankalp Garg (Indian Institute of Technology Delhi)
Mausam (IIT Dehli)

More from the Same Authors

  • 2019 Poster: A Primal Dual Formulation For Deep Learning With Constraints »
    Yatin Nandwani · Abhishek Pathak · Mausam · Parag Singla
  • 2018 : Spotlights 2 »
    Mausam · Ankit Anand · Parag Singla · Tarik Koc · Tim Klinger · Habibeh Naderi · Sungwon Lyu · Saeed Amizadeh · Kshitij Dwivedi · Songpeng Zu · Wei Feng · Balaraman Ravindran · Edouard Pineau · Abdulkadir Celikkanat · Deepak Venugopal