Timezone: »

 
Poster
Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning
Wonjae Kim · Yoonho Lee

Wed Dec 11 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #115
in Applications -- Visual Question Answering »

Without relevant human priors, neural networks may learn uninterpretable features. We propose Dynamics of Attention for Focus Transition (DAFT) as a human prior for machine reasoning. DAFT is a novel method that regularizes attention-based reasoning by modelling it as a continuous dynamical system using neural ordinary differential equations. As a proof of concept, we augment a state-of-the-art visual reasoning model with DAFT. Our experiments reveal that applying DAFT yields similar performance to the original model while using fewer reasoning steps, showing that it implicitly learns to skip unnecessary steps. We also propose a new metric, Total Length of Transition (TLT), which represents the effective reasoning step size by quantifying how much a given model's focus drifts while reasoning about a question. We show that adding DAFT results in lower TLT, demonstrating that our method indeed obeys the human prior towards shorter reasoning paths in addition to producing more interpretable attention maps.

Keywords: Visual Question Answering · Applications · Applications -> Visual Scene Analysis and Interpretation; Deep Learning -> Attention Models; Deep Learning · Memory-Augmented
[ Paper]  [ Poster] 

Author Information

Wonjae Kim (Kakao Corporation)
Yoonho Lee (Kakao Corporation)

More from the Same Authors