Timezone: »
3D shape representations that accommodate learning-based 3D reconstruction are an open problem in machine learning and computer graphics. Previous work on neural 3D reconstruction demonstrated benefits, but also limitations, of point cloud, voxel, surface mesh, and implicit function representations. We introduce \emph{Deformable Tetrahedral Meshes} (DefTet) as a particular parameterization that utilizes volumetric tetrahedral meshes for the reconstruction problem. Unlike existing volumetric approaches, DefTet optimizes for both vertex placement and occupancy, and is differentiable with respect to standard 3D reconstruction loss functions. It is thus simultaneously high-precision, volumetric, and amenable to learning-based neural architectures. We show that it can represent arbitrary, complex topology, is both memory and computationally efficient, and can produce high-fidelity reconstructions with a significantly smaller grid size than alternative volumetric approaches. The predicted surfaces are also inherently defined as tetrahedral meshes, thus do not require post-processing. We demonstrate that DefTetmatches or exceeds both the quality of the previous best approaches and the performance of the fastest ones. Our approach obtains high-quality tetrahedral meshes computed directly from noisy point clouds, and is the first to showcase high-quality 3D results using only a single image as input.
Author Information
Jun Gao (University of Toronto; Nvidia)
Wenzheng Chen (University of Toronto)
Tommy Xiang (University of Toronto)
Alec Jacobson (University of Toronto)
Morgan McGuire (NVIDIA)
Sanja Fidler (University of Toronto)
More from the Same Authors
-
2020 Poster: Variational Amodal Object Completion »
Huan Ling · David Acuna · Karsten Kreis · Seung Wook Kim · Sanja Fidler -
2019 Poster: Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer »
Wenzheng Chen · Huan Ling · Jun Gao · Edward Smith · Jaakko Lehtinen · Alec Jacobson · Sanja Fidler -
2019 Demonstration: Toronto Annotation Suite »
Amlan Kar · Sanja Fidler · Jun Gao · Seung Wook Kim · huan ling -
2018 Poster: A Neural Compositional Paradigm for Image Captioning »
Bo Dai · Sanja Fidler · Dahua Lin -
2017 Poster: Teaching Machines to Describe Images with Natural Language Feedback »
huan ling · Sanja Fidler -
2016 Poster: Proximal Deep Structured Models »
Shenlong Wang · Sanja Fidler · Raquel Urtasun -
2015 Poster: Skip-Thought Vectors »
Jamie Kiros · Yukun Zhu · Russ Salakhutdinov · Richard Zemel · Raquel Urtasun · Antonio Torralba · Sanja Fidler -
2015 Poster: 3D Object Proposals for Accurate Object Class Detection »
Xiaozhi Chen · Kaustav Kundu · Yukun Zhu · Andrew G Berneshawi · Huimin Ma · Sanja Fidler · Raquel Urtasun -
2012 Poster: 3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model »
Sanja Fidler · Sven Dickinson · Raquel Urtasun -
2012 Spotlight: 3D Object Detection and Viewpoint Estimation with a Deformable 3D Cuboid Model »
Sanja Fidler · Sven Dickinson · Raquel Urtasun -
2009 Poster: Evaluating multi-class learning strategies in a generative hierarchical framework for object detection »
Sanja Fidler · Marko Boben · Ales Leonardis