Timezone: »
We present a novel deep architecture and a training strategy to learn a local feature pipeline from scratch, using collections of images without the need for human supervision. To do so we exploit depth and relative camera pose cues to create a virtual target that the network should achieve on one image, provided the outputs of the network for the other image. While this process is inherently non-differentiable, we show that we can optimize the network in a two-branch setup by confining it to one branch, while preserving differentiability in the other. We train our method on both indoor and outdoor datasets, with depth data from 3D sensors for the former, and depth estimates from an off-the-shelf Structure-from-Motion solution for the latter. Our models outperform the state of the art on sparse feature matching on both datasets, while running at 60+ fps for QVGA images.
Author Information
Yuki Ono (SONY)
Eduard Trulls (EPFL)
Pascal Fua (EPFL, Switzerland)
Kwang Moo Yi (University of Victoria)
More from the Same Authors
-
2021 : SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation »
Robin Chan · Krzysztof Lis · Svenja Uhlemeyer · Hermann Blum · Sina Honari · Roland Siegwart · Pascal Fua · Mathieu Salzmann · Matthias Rottmann -
2023 Poster: ISP: Multi-Layered Garment Draping with Implicit Sewing Patterns »
Ren Li · Benoit Guillard · Pascal Fua -
2020 Poster: UCLID-Net: Single View Reconstruction in Object Space »
Benoit Guillard · Edoardo Remelli · Pascal Fua -
2020 Poster: DISK: Learning local features with policy gradient »
Michał Tyszkiewicz · Pascal Fua · Eduard Trulls -
2020 Poster: MeshSDF: Differentiable Iso-Surface Extraction »
Edoardo Remelli · Artem Lukoianov · Stephan Richter · Benoit Guillard · Timur Bagautdinov · Pierre Baque · Pascal Fua -
2020 Spotlight: DISK: Learning local features with policy gradient »
Michał Tyszkiewicz · Pascal Fua · Eduard Trulls -
2020 Spotlight: MeshSDF: Differentiable Iso-Surface Extraction »
Edoardo Remelli · Artem Lukoianov · Stephan Richter · Benoit Guillard · Timur Bagautdinov · Pierre Baque · Pascal Fua -
2019 Demonstration: Real Time CFD simulations with 3D Mesh Convolutional Networks »
Pierre Baque · Pascal Fua · François Fleuret -
2019 Poster: Backpropagation-Friendly Eigendecomposition »
Wei Wang · Zheng Dang · Yinlin Hu · Pascal Fua · Mathieu Salzmann -
2017 Poster: Learning Active Learning from Data »
Ksenia Konyushkova · Raphael Sznitman · Pascal Fua -
2015 Poster: Kullback-Leibler Proximal Variational Inference »
Mohammad Emtiyaz Khan · Pierre Baque · François Fleuret · Pascal Fua -
2013 Poster: Non-Linear Domain Adaptation with Boosting »
Carlos J Becker · Christos M Christoudias · Pascal Fua