Timezone: »
Predicting depth is an essential component in understanding the 3D geometry of a scene. While for stereo images local correspondence suffices for estimation, finding depth relations from a single image is less straightforward, requiring integration of both global and local information from various cues. Moreover, the task is inherently ambiguous, with a large source of uncertainty coming from the overall scale. In this paper, we present a new method that addresses this task by employing two deep network stacks: one that makes a coarse global prediction based on the entire image, and another that refines this prediction locally. We also apply a scale-invariant error to help measure depth relations rather than scale. By leveraging the raw datasets as large sources of training data, our method achieves state-of-the-art results on both NYU Depth and KITTI, and matches detailed depth boundaries without the need for superpixelation.
Author Information
David Eigen (Clarifai)
Christian Puhrsch
Rob Fergus (DeepMind / NYU)
Rob Fergus is an Associate Professor of Computer Science at the Courant Institute of Mathematical Sciences, New York University. He received a Masters in Electrical Engineering with Prof. Pietro Perona at Caltech, before completing a PhD with Prof. Andrew Zisserman at the University of Oxford in 2005. Before coming to NYU, he spent two years as a post-doc in the Computer Science and Artificial Intelligence Lab (CSAIL) at MIT, working with Prof. William Freeman. He has received several awards including a CVPR best paper prize, a Sloan Fellowship & NSF Career award and the IEEE Longuet-Higgins prize.
Related Events (a corresponding poster, oral, or spotlight)
-
2014 Poster: Depth Map Prediction from a Single Image using a Multi-Scale Deep Network »
Thu. Dec 11th 07:00 -- 11:00 PM Room Level 2, room 210D
More from the Same Authors
-
2021 : Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learning »
Denis Yarats · Rob Fergus · Alessandro Lazaric · Lerrel Pinto -
2022 : Collaborating with language models for embodied reasoning »
Ishita Dasgupta · Christine Kaeser-Chen · Kenneth Marino · Arun Ahuja · Sheila Babayan · Felix Hill · Rob Fergus -
2022 : Collaborating with language models for embodied reasoning »
Ishita Dasgupta · Christine Kaeser-Chen · Kenneth Marino · Arun Ahuja · Sheila Babayan · Felix Hill · Rob Fergus -
2023 Poster: NetHack is Hard to Hack »
Ulyana Piterbarg · Lerrel Pinto · Rob Fergus -
2022 Poster: Learning to Navigate Wikipedia by Taking Random Walks »
Manzil Zaheer · Kenneth Marino · Will Grathwohl · John Schultz · Wendy Shang · Sheila Babayan · Arun Ahuja · Ishita Dasgupta · Christine Kaeser-Chen · Rob Fergus -
2021 Poster: Automatic Data Augmentation for Generalization in Reinforcement Learning »
Roberta Raileanu · Maxwell Goldstein · Denis Yarats · Ilya Kostrikov · Rob Fergus -
2020 : Contributed Talk - Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences »
Alexander Rives · Siddharth Goyal · Joshua Meier · Zeming Lin · Demi Guo · Myle Ott · Larry Zitnick · Rob Fergus -
2016 Poster: Learning Multiagent Communication with Backpropagation »
Sainbayar Sukhbaatar · arthur szlam · Rob Fergus -
2014 Poster: Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation »
Emily Denton · Wojciech Zaremba · Joan Bruna · Yann LeCun · Rob Fergus -
2014 Poster: Learning to Discover Efficient Mathematical Identities »
Wojciech Zaremba · Karol Kurach · Rob Fergus -
2014 Spotlight: Learning to Discover Efficient Mathematical Identities »
Wojciech Zaremba · Karol Kurach · Rob Fergus -
2013 Tutorial: Deep Learning for Computer Vision »
Rob Fergus -
2011 Workshop: Machine Learning meets Computational Photography »
Michael Hirsch · Stefan Harmeling · Rob Fergus · Peyman Milanfar -
2011 Poster: Facial Expression Transfer with Input-Output Temporal Restricted Boltzmann Machines »
Matthew D Zeiler · Graham Taylor · Leonid Sigal · Iain Matthews · Rob Fergus -
2011 Session: Spotlight Session 1 »
Rob Fergus -
2010 Session: Oral Session 17 »
Rob Fergus -
2010 Poster: Pose-Sensitive Embedding by Nonlinear NCA Regression »
Graham Taylor · Rob Fergus · George Williams · Ian Spiro · Christoph Bregler -
2009 Poster: Fast Image Deconvolution using Hyper-Laplacian Priors »
Dilip Krishnan · Rob Fergus -
2009 Spotlight: Fast Image Deconvolution using Hyper-Laplacian Priors »
Dilip Krishnan · Rob Fergus -
2009 Poster: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2009 Oral: Semi-Supervised Learning in Gigantic Image Collections »
Rob Fergus · Yair Weiss · Antonio Torralba -
2008 Poster: Spectral Hashing »
Yair Weiss · Antonio Torralba · Rob Fergus -
2007 Spotlight: Object Recognition by Scene Alignment »
Bryan C Russell · Antonio Torralba · Ce Liu · Rob Fergus · William Freeman -
2007 Poster: Object Recognition by Scene Alignment »
Bryan C Russell · Antonio Torralba · Ce Liu · Rob Fergus · William Freeman