Timezone: »

 
Poster
Neural Non-Rigid Tracking
Aljaz Bozic · Pablo Palafox · Michael Zollhöfer · Angela Dai · Justus Thies · Matthias Niessner

Tue Dec 08 09:00 AM -- 11:00 AM (PST) @ Poster Session 1 #477
in Poster Session 2 »

We introduce a novel, end-to-end learnable, differentiable non-rigid tracker that enables state-of-the-art non-rigid reconstruction by a learned robust optimization. Given two input RGB-D frames of a non-rigidly moving object, we employ a convolutional neural network to predict dense correspondences and their confidences. These correspondences are used as constraints in an as-rigid-as-possible (ARAP) optimization problem. By enabling gradient back-propagation through the weighted non-linear least squares solver, we are able to learn correspondences and confidences in an end-to-end manner such that they are optimal for the task of non-rigid tracking. Under this formulation, correspondence confidences can be learned via self-supervision, informing a learned robust optimization, where outliers and wrong correspondences are automatically down-weighted to enable effective tracking. Compared to state-of-the-art approaches, our algorithm shows improved reconstruction performance, while simultaneously achieving 85 times faster correspondence prediction than comparable deep-learning based methods.

Author Information

Aljaz Bozic (Technical University Munich)
Pablo Palafox (Technical University Munich)
Michael Zollhöfer (Stanford University)
Angela Dai (Technical University of Munich)
Justus Thies (Technical University of Munich)
Matthias Niessner (Technical University of Munich)

More from the Same Authors