Timezone: »

 
Generalization and Translatability in Emergent Communication via Informational Constraints
Mycal Tucker · Roger Levy · Julie A Shah · Noga Zaslavsky

Sat Dec 03 09:40 AM -- 09:43 AM (PST) @
in Information-Theoretic Principles in Cognitive Systems »
Event URL: https://openreview.net/forum?id=yf8suFtNZ5v »

Traditional emergent communication (EC) methods often fail to generalize to novel settings or align with representations of natural language. Here, we show how controlling the Information Bottleneck (IB) tradeoff between complexity and informativeness (a principle thought to guide human languages) helps to address both of these problems in EC. Using VQ-VIB, a recent method for training EC agents while controlling the IB tradeoff, we find that: (1) increasing pressure for informativeness, which encourages agents to develop a shared understanding beyond task-specific needs, leads to better generalization to more challenging tasks and novel inputs; (2) VQ-VIB agents develop an EC space that encodes some semantic similarities and facilitates open-domain communication, similar to word embeddings in natural language; and (3) when translating between English and EC, greater complexity leads to improved performance of teams of simulated English speakers and trained VQ-VIB listeners, but only up to a threshold corresponding to the English complexity. These results indicate the importance of informational constraints for improving self-play performance and human-agent interaction.

Author Information

Mycal Tucker (Massachusetts Institute of Technology)
Roger Levy (Massachusetts Institute of Technology)
Julie A Shah (MIT)
Noga Zaslavsky (MIT)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors