`

Timezone: »

 
Poster
Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data
Michael Cogswell · Jiasen Lu · Rishabh Jain · Stefan Lee · Devi Parikh · Dhruv Batra

Mon Dec 07 09:00 PM -- 11:00 PM (PST) @ Poster Session 0 #63
in Poster Session 1 »

Can we develop visually grounded dialog agents that can efficiently adapt to new tasks without forgetting how to talk to people? Such agents could leverage a larger variety of existing data to generalize to a new task, minimizing expensive data collection and annotation. In this work, we study a setting we call "Dialog without Dialog", which requires agents to develop visually grounded dialog models that can adapt to new tasks without language level supervision. By factorizing intention and language, our model minimizes linguistic drift after fine-tuning for new tasks. We present qualitative results, automated metrics, and human studies that all show our model can adapt to new tasks and maintain language quality. Baselines either fail to perform well at new tasks or experience language drift, becoming unintelligible to humans. Code has been made available at: https://github.com/mcogswell/dialogwithoutdialog.

Author Information

Michael Cogswell (SRI International)
Jiasen Lu (Allen Institute of Artificial Intelligence)
Rishabh Jain (Georgia Tech)
Stefan Lee (Oregon State University)
Devi Parikh (Georgia Tech / Facebook AI Research (FAIR))
Dhruv Batra (Georgia Tech / Facebook AI Research (FAIR))

More from the Same Authors