Timezone: »
In images of complex scenes, objects are often occluding each other which makes perception tasks such as object detection and tracking, or robotic control tasks such as planning, challenging. To facilitate downstream tasks, it is thus important to reason about the full extent of objects, i.e., seeing behind occlusion, typically referred to as amodal instance completion. In this paper, we propose a variational generative framework for amodal completion, referred to as AMODAL-VAE, which does not require any amodal labels at training time, as it is able to utilize widely available object instance masks. We showcase our approach on the downstream task of scene editing where the user is presented with interactive tools to complete and erase objects in photographs. Experiments on complex street scenes demonstrate state-of-the-art performance in amodal mask completion and showcase high-quality scene editing results. Interestingly, a user study shows that humans prefer object completions inferred by our model to the human-labeled ones.
Author Information
Huan Ling (University of Toronto, NVIDIA)
David Acuna (University of Toronto, Nvidia, Vector Institute)
Karsten Kreis (NVIDIA)
Seung Wook Kim (University of Toronto)
Sanja Fidler (University of Toronto)
More from the Same Authors
-
2020 Poster: Learning Deformable Tetrahedral Meshes for 3D Reconstruction »
Jun Gao · Wenzheng Chen · Tommy Xiang · Alec Jacobson · Morgan McGuire · Sanja Fidler -
2019 Poster: Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer »
Wenzheng Chen · Huan Ling · Jun Gao · Edward Smith · Jaakko Lehtinen · Alec Jacobson · Sanja Fidler -
2019 Demonstration: Toronto Annotation Suite »
Amlan Kar · Sanja Fidler · Jun Gao · Seung Wook Kim · huan ling -
2018 Poster: A Neural Compositional Paradigm for Image Captioning »
Bo Dai · Sanja Fidler · Dahua Lin -
2017 Poster: Teaching Machines to Describe Images with Natural Language Feedback »
huan ling · Sanja Fidler -
2016 Poster: Proximal Deep Structured Models »
Shenlong Wang · Sanja Fidler · Raquel Urtasun -
2015 Poster: Skip-Thought Vectors »
Jamie Kiros · Yukun Zhu · Russ Salakhutdinov · Richard Zemel · Raquel Urtasun · Antonio Torralba · Sanja Fidler -
2015 Poster: 3D Object Proposals for Accurate Object Class Detection »
Xiaozhi Chen · Kaustav Kundu · Yukun Zhu · Andrew G Berneshawi · Huimin Ma · Sanja Fidler · Raquel Urtasun -
2012 Poster: 3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model »
Sanja Fidler · Sven Dickinson · Raquel Urtasun -
2012 Spotlight: 3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model »
Sanja Fidler · Sven Dickinson · Raquel Urtasun -
2009 Poster: Evaluating multi-class learning strategies in a generative hierarchical framework for object detection »
Sanja Fidler · Marko Boben · Ales Leonardis