Reasoning about Action Preconditions with Programs

One of the fundamental skills required for an agent acting in an environment to complete tasks is the ability to understand what actions are plausible at any given point. This work explores a novel use of code representations to reason about action preconditions for sequential decision making tasks. Code representations offer the flexibility to model procedural activities and associated constraints as well as the ability to execute and verify constraint satisfaction. Leveraging code representations, we decompose the problem of learning an agent policy for sequential decision making tasks into the sub-problems of precondition inference and action prediction. We show that these sub-problems can be formulated as code-completion problems and exploit pre-trained code understanding models to tackle them. We demonstrate that the proposed code representation coupled with our novel precondition-aware action prediction strategy outperforms prior policy learning approaches in a few-shot learning setting across task-oriented dialog and embodied textworld benchmarks.