Actively learning a Bayesian matrix fusion model with deep side information

High-dimensional deep neural network representations of images and concepts can bealigned to predict human annotations of diverse stimuli. However, such alignment requiresthe costly collection of behavioral responses, such that, in practice, the deep-featurespaces are only ever sparsely sampled. Here, we propose an active learning approach toadaptively sample experimental stimuli to efficiently learn a Bayesian matrix factorizationmodel with deep side information. We observe a significant efficiency gain over a passivebaseline. Furthermore, with a sequential batched sampling strategy, the algorithm is applicablenot only to small datasets collected from traditional laboratory experiments butalso to settings where large-scale crowdsourced data collection is needed to accurately alignthe high-dimensional deep feature representations derived from pre-trained networks. Thisprovides cost-effective solutions for collecting and generating quality-assured predictions inlarge-scale behavioral and cognitive studies.