Timezone: »

 
Workshop
Workshop on Deep Learning and Inverse Problems
Reinhard Heckel · Paul Hand · Richard Baraniuk · Lenka Zdeborová · Soheil Feizi

Fri Dec 11 07:30 AM -- 04:00 PM (PST) @ None
Event URL: https://deep-inverse.org/ »

Learning-based methods, and in particular deep neural networks, have emerged as highly successful and universal tools for image and signal recovery and restoration. They achieve state-of-the-art results on tasks ranging from image denoising, image compression, and image reconstruction from few and noisy measurements. They are starting to be used in important imaging technologies, for example in GEs newest computational tomography scanners and in the newest generation of the iPhone.

The field has a range of theoretical and practical questions that remain unanswered. In particular, learning and neural network-based approaches often lack the guarantees of traditional physics-based methods. Further, while superior on average, learning-based methods can make drastic reconstruction errors, such as hallucinating a tumor in an MRI reconstruction or turning a pixelated picture of Obama into a white male.

This virtual workshop aims at bringing together theoreticians and practitioners in order to chart out recent advances and discuss new directions in deep neural network-based approaches for solving inverse problems in the imaging sciences and beyond. NeurIPS, with its visibility and attendance by experts in machine learning, offers the ideal frame for this exchange of ideas. We will use this virtual format to make this topic accessible to a broader audience than the in-person meeting is able to as described below.

Fri 7:30 a.m. - 7:55 a.m.
Video

This session consists of a 15-minute talk and a 10 minute Q/A geared toward newcomers to the field, introducing them to the major questions and approaches related to deep learning and inverse problems.

Reinhard Heckel, Paul Hand
Fri 7:55 a.m. - 8:00 a.m.
Opening Remarks
Reinhard Heckel, Paul Hand, Soheil Feizi, Lenka Zdeborová, Richard Baraniuk
Fri 8:00 a.m. - 8:30 a.m.
Victor Lempitsky - Generative Models for Landscapes and Avatars (Invited talk and Q&A) Video
Victor Lempitsky
Fri 8:30 a.m. - 9:00 a.m.
Thomas Pock - Variational Networks (Invited talk and Q&A) Video
Thomas Pock
Fri 9:00 a.m. - 9:15 a.m.
Video

Reliable medical image recovery is crucial for accurate patient diagnoses, but little prior work has centered on quantifying uncertainty when using non-transparent deep learning approaches to reconstruct high-quality images from limited measured data. In this study, we develop methods to address these concerns, utilizing a VAE as a probabilistic recovery algorithm for pediatric knee MR imaging. Through our use of SURE, which examines the end-to-end network Jacobian, we demonstrate a new and rigorous metric for assessing risk in medical image recovery that applies universally across model architectures.

Vineet Edupuganti
Fri 9:15 a.m. - 9:30 a.m.
Video

We tackle a challenging blind image denoising problem, in which only single distinct noisy images are available for training a denoiser, and no information about noise is known, except for it being zero-mean, additive, and independent of the clean image. In such a setting, which often occurs in practice, it is not possible to train a denoiser with the standard discriminative training or with the recently developed Noise2Noise (N2N) training; the former requires the underlying clean image for the given noisy image, and the latter requires two independently realized noisy image pair for a clean image. To that end, we propose GAN2GAN (Generated-Artificial-Noise to Generated-Artificial-Noise) method that first learns a generative model that can 1) simulate the noise in the given noisy images and 2) generate a rough, noisy estimates of the clean images, then 3) iteratively trains a denoiser with subsequently synthesized noisy image pairs (as in N2N), obtained from the generative model. In results, we show the denoiser trained with our GAN2GAN achieves an impressive denoising performance on both synthetic and real-world datasets for the blind denoising setting.

Sungmin Cha
Fri 9:30 a.m. - 10:00 a.m.

Visit the Gather.town to discuss with speakers and other attendees.

Fri 10:00 a.m. - 10:30 a.m.
Rebecca Willett - Model Adaptation for Inverse Problems in Imaging (Invited talk and Q&A) Video
Rebecca Willett
Fri 10:30 a.m. - 11:00 a.m.
Stefano Emron - Generative Modeling via Denoising (Invited talk and Q&A) Video
Stefano Ermon
Fri 11:00 a.m. - 11:15 a.m.
Video

We characterize the measurement complexity of compressed sensing of signals drawn from a known prior distribution, even when the support of the prior is the entire space (rather than, say, sparse vectors). We show for Gaussian measurements and \emph{any} prior distribution on the signal, that the conditional resampling estimator achieves near-optimal recovery guarantees. Moreover, this result is robust to model mismatch, as long as the distribution estimate (e.g., from an invertible generative model) is close to the true distribution in Wasserstein distance. We implement the conditional resampling estimator for deep generative priors using Langevin dynamics, and empirically find that it produces accurate estimates with more diversity than MAP.

Ajil Jalal
Fri 11:15 a.m. - 11:30 a.m.
Chris Metzler - Approximate Message Passing (AMP) Algorithms for Computational Imaging (Invited Talk and Q&A) Video
Christopher Metzler
Fri 11:30 a.m. - 12:00 p.m.

Visit the Gather.town to discuss with speakers and other attendees.

Fri 1:00 p.m. - 2:00 p.m.

Visit the gather.town to see the posters.

Fri 2:00 p.m. - 2:30 p.m.
Peyman Milanfar - Denoising as Building Block Theory and Applications (Invited talk and Q&A) Video
Peyman Milanfar
Fri 2:30 p.m. - 3:00 p.m.
Rachel Ward (Invited talk and Q&A)
Rachel Ward
Fri 3:00 p.m. - 3:30 p.m.
Larry Zitnick - fastMRI (Invited talk and Q&A) Video
Larry Zitnick
Fri 3:30 p.m. - 4:00 p.m.

Visit the Gather.town to discuss with speakers and other attendees.

Author Information

Reinhard Heckel (TUM / Rice University)
Paul Hand (Northeastern University)
Richard Baraniuk (Rice University)
Lenka Zdeborová (CEA)
Soheil Feizi (University of Maryland)

More from the Same Authors