Timezone: »

 
Poster
Computational Mirrors: Blind Inverse Light Transport by Deep Matrix Factorization
Miika Aittala · Prafull Sharma · Lukas Murmann · Adam Yedidia · Gregory Wornell · Bill Freeman · Fredo Durand

Thu Dec 12 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #82
in Applications -- Computational Photography »

We recover a video of the motion taking place in a hidden scene by observing changes in indirect illumination in a nearby uncalibrated visible region. We solve this problem by factoring the observed video into a matrix product between the unknown hidden scene video and an unknown light transport matrix. This task is extremely ill-posed, as any non-negative factorization will satisfy the data. Inspired by recent work on the Deep Image Prior, we parameterize the factor matrices using randomly initialized convolutional neural networks trained in a one-off manner, and show that this results in decompositions that reflect the true motion in the hidden scene.

Keywords: Generative Models · Computational Photography · Applications · Applications -> Computer Vision; Applications -> Matrix and Tensor Factorization; Deep Learning

Author Information

Miika Aittala (MIT CSAIL / NVIDIA)
Prafull Sharma (MIT)
Lukas Murmann (Massachusetts Institute of Technology)
Adam Yedidia (Massachusetts Institute of Technology)
Gregory Wornell (MIT)
Bill Freeman (MIT/Google)
Fredo Durand (MIT)

More from the Same Authors