Timezone: »

 
Poster
Lost in Latent Space: Examining failures of disentangled models at combinatorial generalisation
Milton Montero · Jeffrey Bowers · Rui Ponte Costa · Casimir Ludwig · Gaurav Malhotra

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #738
in Poster Session 1 »

Recent research has shown that generative models with highly disentangled representations fail to generalise to unseen combination of generative factor values. These findings contradict earlier research which showed improved performance in out-of-training distribution settings when compared to entangled representations. Additionally, it is not clear if the reported failures are due to (a) encoders failing to map novel combinations to the proper regions of the latent space, or (b) novel combinations being mapped correctly but the decoder is unable to render the correct output for the unseen combinations. We investigate these alternatives by testing several models on a range of datasets and training settings. We find that (i) when models fail, their encoders also fail to map unseen combinations to correct regions of the latent space and (ii) when models succeed, it is either because the test conditions do not exclude enough examples, or because excluded cases involve combinations of object properties with it's shape. We argue that to generalise properly, models not only need to capture factors of variation, but also understand how to invert the process that causes the visual stimulus.

Keywords: Generative Models · Representation Learning · Disentanglement · Combinatorial Generalisation

Author Information

Milton Montero (University of Bristol)
Jeffrey Bowers (University of Bristol)
Rui Ponte Costa (University of Bristol)
Casimir Ludwig
Gaurav Malhotra (University of Bristol)

More from the Same Authors