Skip to yearly menu bar Skip to main content

Invited Talk
Workshop: Learning by Instruction

Mapping Navigation Instructions to Continuous Control

Yoav Artzi


Natural language understanding in grounded interactive scenarios is tightly coupled with the actions the system generates. The action space used determines much of the complexity of the problem and the type of reasoning required. In this talk, I will describe our approach to learning to map instructions and observations to continuous control of a realistic quadcopter drone. This scenario raises new challenging questions including how can we use demonstrations to learn to bridge the gap between the high-level concepts of language and low-level robot controls? And how do we design models that continuously observe, control, and react to a rapidly changing environment? This work uses a new publicly available evaluation benchmark.

Live content is unavailable. Log in and register to view live content