Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Generalization in Planning (GenPlan '23)

Relating Goal and Environmental Complexity for Improved Task Transfer: Initial Results

Sunandita Patra · Paul Rademacher · Kristen Jacobson · Kyle Hassold · Onur Kulaksizoglu · Laura Hiatt · Mark Roberts · Dana Nau

Keywords: [ transfer learning ] [ Reinforcement Learning ] [ hierarchical planning ] [ planning and learning ] [ curriculum learning ]


Abstract:

The complexity of an environment and the difficulty of an actor's goals both impact transfer learning in Reinforcement Learning (RL). Yet, few works have examined using the environment and goals in tandem to generate a learning curriculum that improves transfer. To explore this relationship, we introduce a task graph that quantifies the environment complexity using environment descriptors and the goal difficulty using goal descriptors; edges in the task graph indicate a change in the environment or the goal. We use the task graph in two sets of studies. First, we evaluate the task graph in two synthetic environments where we control environment and goal complexity. Second, we introduce an algorithm that generates a Task-Graph Curriculum to train policies using the task graph. In a delivery environment with up to ten skills, we demonstrate that a planner can execute these trained policies to achieve long-horizon goals in increasingly complex environments. Our results demonstrate that (1) the task graph promotes skill transfer in the synthetic environments and (2) the Task-Graph Curriculum trains nearly perfect policies and does so significantly faster than learning a policy from scratch.

Chat is not available.