Timezone: »

 
Neural Structure Mapping For Learning Abstract Visual Analogies
Shashank Shekhar · Graham Taylor

Mon Dec 13 09:00 AM -- 10:00 AM (PST) @
in Shared Visual Representations in Human and Machine Intelligence »
Event URL: https://openreview.net/forum?id=l-TLGjxwajn »

Building conceptual abstractions from sensory information and then reasoning about them is central to human intelligence. Abstract reasoning both relies on, and is facilitated by, our ability to make analogies about concepts from known domains to novel domains. Structure Mapping Theory of human analogical reasoning posits that analogical mappings rely on (higher-order) relations and not on the sensory content of the domain. This enables humans to reason systematically about novel domains, a problem with which machine learning (ML) models tend to struggle. We introduce a two-stage neural framework, which we label Neural Structure Mapping (NSM), to learn visual analogies from Raven's Progressive Matrices, an abstract visual reasoning test of fluid intelligence. Our framework uses (1) a multi-task visual relationship encoder to extract constituent concepts from raw visual input in the source domain, and (2) a neural module net analogy inference engine to reason compositionally about the inferred relation in the target domain. Our NSM approach (a) isolates the relational structure from the source domain with high accuracy, and (b) successfully utilizes this structure for analogical reasoning in the target domain.

Author Information

Shashank Shekhar (Facebook AI Research)
Shashank Shekhar

I am an AI Resident at FAIR, Meta AI at Montreal, Canada. Prior to this, I was a research master's student at the University of Guelph and Vector Institute in Ontario, Canada. My areas of interest are self-supervised learning, representation learning, and computer vision.

Graham Taylor (University of Guelph / Vector Institute)

More from the Same Authors