Skip to yearly menu bar Skip to main content

Workshop: Foundation Models for Decision Making

Language Conditioned Semantic Search Based Policy for Robotic Manipulation Tasks

Jannik Sheikh · Andrew Melnik · Gora Chand Nandi · Robert Haschke

[ ] [ Project Page ]
presentation: Foundation Models for Decision Making
Fri 15 Dec 6:15 a.m. PST — 3:30 p.m. PST


Solving various robotic manipulation tasks intelligently is a topic of great interest. Traditional reinforcement learning and imitation learning approaches require policy learning utilizing complex strategies which are difficult to generalize well. In this work, we propose a language conditioned semantic search based method in the available demonstration dataset of state-action trajectories to produce an on the fly search-based policy. Here we directly acquire actions from the most similar manipulation trajectories found in the dataset. Our approach surpasses the performance of previous best methods on the CALVIN benchmark and exhibits strong zero-shot adaptation capabilities. This holds great potential for expanding the use of our on the fly search-based policy approach to tasks typically addressed by Imitation Learning or Reinforcement Learning-based policies.

Chat is not available.