Timezone: »

 
Poster
SURGE: Surface Regularized Geometry Estimation from a Single Image
Peng Wang · Xiaohui Shen · Bryan Russell · Scott Cohen · Brian Price · Alan Yuille

Tue Dec 06 09:00 AM -- 12:30 PM (PST) @ Area 5+6+7+8 #63
in Tuesday Posters »

This paper introduces an approach to regularize 2.5D surface normal and depth predictions at each pixel given a single input image. The approach infers and reasons about the underlying 3D planar surfaces depicted in the image to snap predicted normals and depths to inferred planar surfaces, all while maintaining fine detail within objects. Our approach comprises two components: (i) a fourstream convolutional neural network (CNN) where depths, surface normals, and likelihoods of planar region and planar boundary are predicted at each pixel, followed by (ii) a dense conditional random field (DCRF) that integrates the four predictions such that the normals and depths are compatible with each other and regularized by the planar region and planar boundary information. The DCRF is formulated such that gradients can be passed to the surface normal and depth CNNs via backpropagation. In addition, we propose new planar wise metrics to evaluate geometry consistency within planar surfaces, which are more tightly related to dependent 3D editing applications. We show that our regularization yields a 30% relative improvement in planar consistency on the NYU v2 dataset.

Keywords: Deep Learning or Neural Networks · (Application) Computer Vision · Structured Prediction

Author Information

Peng Wang (UCLA)
Xiaohui Shen (Adobe Research)
Bryan Russell (Adobe)
Scott Cohen (Adobe Research)
Brian Price (Adobe)
Alan Yuille (JHU)

More from the Same Authors