Timezone: »

 
Poster
Face Reconstruction from Voice using Generative Adversarial Networks
Yandong Wen · Bhiksha Raj · Rita Singh

Tue Dec 10 05:30 PM -- 07:30 PM (PST) @ East Exhibition Hall B + C #67
in Applications -- Body Pose, Face, and Gesture Analysis »

Voice profiling aims at inferring various human parameters from their speech, e.g. gender, age, etc. In this paper, we address the challenge posed by a subtask of voice profiling - reconstructing someone's face from their voice. The task is designed to answer the question: given an audio clip spoken by an unseen person, can we picture a face that has as many common elements, or associations as possible with the speaker, in terms of identity?

To address this problem, we propose a simple but effective computational framework based on generative adversarial networks (GANs). The network learns to generate faces from voices by matching the identities of generated faces to those of the speakers, on a training set. We evaluate the performance of the network by leveraging a closely related task - cross-modal matching. The results show that our model is able to generate faces that match several biometric characteristics of the speaker, and results in matching accuracies that are much better than chance. The code is publicly available in https://github.com/cmu-mlsp/reconstructingfacesfrom_voices

Keywords: Adversarial Networks · Generative Models · Deep Learning · Applications -> Body Pose, Face, and Gesture Analysis; Applications -> Computer Vision; Deep Learning
[ Paper]  [ Poster] 

Author Information

Yandong Wen (Carnegie Mellon University)
Bhiksha Raj (Carnegie Mellon University)
Rita Singh (Carnegie Mellon University)

More from the Same Authors