Timezone: »

Mixture-of-experts VAEs can disregard unimodal variation in surjective multimodal data
Jannik Wolff · Tassilo Klein · Moin Nabi · Rahul G Krishnan · Shinichi Nakajima
Event URL: https://openreview.net/forum?id=Hl-7kChWA4U »

Machine learning systems are often deployed in domains that entail data from multiple modalities, for example, phenotypic and genotypic characteristics describe patients in healthcare. Previous works have developed variational autoencoders (VAEs) that generate multimodal data. We consider surjective data, where single datapoints from one modality (such as labels) describe multiple datapoints from another modality (such as images). We theoretically and empirically demonstrate that multimodal VAEs with mixture of experts posterior can struggle to capture unimodal variability in surjective data.

Author Information

Jannik Wolff (TU Berlin)
Tassilo Klein (SAP SE)
Moin Nabi (SAP SE)
Rahul G Krishnan (New York University)
Shinichi Nakajima (TU Berlin)

More from the Same Authors

  • 2018 : Poster presentations »
    Simon Wiedemann · Huan Wang · Ivan Zhang · Chong Wang · Mohammad Javad Shafiee · Rachel Manzelli · Wenbing Huang · Tassilo Klein · Lifu Zhang · Ashutosh Adhikari · Faisal Qureshi · Giuseppe Castiglione
  • 2017 : Poster Session 2 »
    Farhan Shafiq · Antonio Tomas Nevado Vilchez · Takato Yamada · Sakyasingha Dasgupta · Robin Geyer · Moin Nabi · Crefeda Rodrigues · Edoardo Manino · Alexantrou Serb · Miguel A. Carreira-Perpinan · Kar Wai Lim · Bryan Kian Hsiang Low · Rohit Pandey · Marie C White · Pavel Pidlypenskyi · Xue Wang · Christine Kaeser-Chen · Michael Zhu · Suyog Gupta · Sam Leroux
  • 2015 Poster: Barrier Frank-Wolfe for Marginal Inference »
    Rahul G Krishnan · Simon Lacoste-Julien · David Sontag