Timezone: »
Many computer vision tasks rely on labeled data. Rapid progress in generative modeling has led to the ability to synthesize photorealistic images. However, controlling specific aspects of the generation process such that the data can be used for supervision of downstream tasks remains challenging. In this paper we propose a novel generative model for images of faces, that is capable of producing high-quality images under fine-grained control over eye gaze and head orientation angles. This requires the disentangling of many appearance related factors including gaze and head orientation but also lighting, hue etc. We propose a novel architecture which learns to discover, disentangle and encode these extraneous variations in a self-learned manner. We further show that explicitly disentangling task-irrelevant factors results in more accurate modelling of gaze and head orientation. A novel evaluation scheme shows that our method improves upon the state-of-the-art in redirection accuracy and disentanglement between gaze direction and head orientation changes. Furthermore, we show that in the presence of limited amounts of real-world training data, our method allows for improvements in the downstream task of semi-supervised cross-dataset gaze estimation. Please check our project page at: https://ait.ethz.ch/projects/2020/STED-gaze/
Author Information
Yufeng Zheng (ETH Zurich)
Seonwook Park (ETH Zurich)
Xucong Zhang (ETH Zurich)
Shalini De Mello (NVIDIA)

Shalini De Mello is a Principal Research Scientist and Research Lead in the Learning and Perception Research group at NVIDIA, which she joined in 2013. Her research interests are in human-centric vision (face and gaze analysis) and in data-efficient (synth2real, low-shot, self-supervised and multimodal) machine learning. She has co-authored 48 peer-reviewed publications and holds 38 patents. Her inventions have contributed to several NVIDIA products, including DriveIX and Maxine. Previously, she has worked at Texas Instruments and AT&T Laboratories. She received her Doctoral degree in Electrical and Computer Engineering from the University of Texas at Austin.
Otmar Hilliges (ETH Zurich)
More from the Same Authors
-
2021 : TempoRL: Temporal Priors for Exploration in Off-Policy Reinforcement Learning »
Marco Bagatella · Sammy Christen · Otmar Hilliges -
2022 : Exploiting Human Interactions to Learn Human Attention »
Shalini De Mello -
2020 : Contributed Talk: Learning Functionally Decomposed Hierarchies for Continuous Control Tasks with Path Planning »
Sammy Christen · Lukas Jendele · Emre Aksan · Otmar Hilliges -
2020 Poster: Online Adaptation for Consistent Mesh Reconstruction in the Wild »
Xueting Li · Sifei Liu · Shalini De Mello · Kihwan Kim · Xiaolong Wang · Ming-Hsuan Yang · Jan Kautz -
2020 Poster: CoSE: Compositional Stroke Embeddings »
Emre Aksan · Thomas Deselaers · Andrea Tagliasacchi · Otmar Hilliges -
2019 Poster: Joint-task Self-supervised Learning for Temporal Correspondence »
Xueting Li · Sifei Liu · Shalini De Mello · Xiaolong Wang · Jan Kautz · Ming-Hsuan Yang -
2017 Poster: Learning Affinity via Spatial Propagation Networks »
Sifei Liu · Shalini De Mello · Jinwei Gu · Guangyu Zhong · Ming-Hsuan Yang · Jan Kautz