Workshop
Workshop on Deep Learning and Inverse Problems
Reinhard Heckel · Paul Hand · Richard Baraniuk · Lenka Zdeborová · Soheil Feizi
Learning-based methods, and in particular deep neural networks, have emerged as highly successful and universal tools for image and signal recovery and restoration. They achieve state-of-the-art results on tasks ranging from image denoising, image compression, and image reconstruction from few and noisy measurements. They are starting to be used in important imaging technologies, for example in GEs newest computational tomography scanners and in the newest generation of the iPhone.
The field has a range of theoretical and practical questions that remain unanswered. In particular, learning and neural network-based approaches often lack the guarantees of traditional physics-based methods. Further, while superior on average, learning-based methods can make drastic reconstruction errors, such as hallucinating a tumor in an MRI reconstruction or turning a pixelated picture of Obama into a white male.
This virtual workshop aims at bringing together theoreticians and practitioners in order to chart out recent advances and discuss new directions in deep neural network-based approaches for solving inverse problems in the imaging sciences and beyond. NeurIPS, with its visibility and attendance by experts in machine learning, offers the ideal frame for this exchange of ideas. We will use this virtual format to make this topic accessible to a broader audience than the in-person meeting is able to as described below.
Schedule
Fri 7:30 a.m. - 7:55 a.m.
|
Newcomer presentation
(
Talk and Q&A
)
>
SlidesLive Video This session consists of a 15-minute talk and a 10 minute Q/A geared toward newcomers to the field, introducing them to the major questions and approaches related to deep learning and inverse problems. |
Reinhard Heckel · Paul Hand 🔗 |
Fri 7:55 a.m. - 8:00 a.m.
|
Opening Remarks
(
Opening Remarks
)
>
|
Reinhard Heckel · Paul Hand · Soheil Feizi · Lenka Zdeborová · Richard Baraniuk 🔗 |
Fri 8:00 a.m. - 8:30 a.m.
|
Victor Lempitsky - Generative Models for Landscapes and Avatars
(
Invited talk and Q&A
)
>
SlidesLive Video |
Victor Lempitsky 🔗 |
Fri 8:30 a.m. - 9:00 a.m.
|
Thomas Pock - Variational Networks
(
Invited talk and Q&A
)
>
SlidesLive Video |
Thomas Pock 🔗 |
Fri 9:00 a.m. - 9:15 a.m.
|
Risk Quantification in Deep MRI Reconstruction
(
Contributed Talk and Q&A
)
>
SlidesLive Video Reliable medical image recovery is crucial for accurate patient diagnoses, but little prior work has centered on quantifying uncertainty when using non-transparent deep learning approaches to reconstruct high-quality images from limited measured data. In this study, we develop methods to address these concerns, utilizing a VAE as a probabilistic recovery algorithm for pediatric knee MR imaging. Through our use of SURE, which examines the end-to-end network Jacobian, we demonstrate a new and rigorous metric for assessing risk in medical image recovery that applies universally across model architectures. |
Vineet Edupuganti 🔗 |
Fri 9:15 a.m. - 9:30 a.m.
|
GAN2GAN: Generative Noise Learning for Blind Denoising with Single Noisy Images
(
Contributed Talk and Q&A
)
>
SlidesLive Video We tackle a challenging blind image denoising problem, in which only single distinct noisy images are available for training a denoiser, and no information about noise is known, except for it being zero-mean, additive, and independent of the clean image. In such a setting, which often occurs in practice, it is not possible to train a denoiser with the standard discriminative training or with the recently developed Noise2Noise (N2N) training; the former requires the underlying clean image for the given noisy image, and the latter requires two independently realized noisy image pair for a clean image. To that end, we propose GAN2GAN (Generated-Artificial-Noise to Generated-Artificial-Noise) method that first learns a generative model that can 1) simulate the noise in the given noisy images and 2) generate a rough, noisy estimates of the clean images, then 3) iteratively trains a denoiser with subsequently synthesized noisy image pairs (as in N2N), obtained from the generative model. In results, we show the denoiser trained with our GAN2GAN achieves an impressive denoising performance on both synthetic and real-world datasets for the blind denoising setting. |
Sungmin Cha 🔗 |
Fri 9:30 a.m. - 10:00 a.m.
|
Discussion
(
Break and Discussion
)
>
link
Visit the Gather.town to discuss with speakers and other attendees. |
🔗 |
Fri 10:00 a.m. - 10:30 a.m.
|
Rebecca Willett - Model Adaptation for Inverse Problems in Imaging
(
Invited talk and Q&A
)
>
SlidesLive Video |
Rebecca Willett 🔗 |
Fri 10:30 a.m. - 11:00 a.m.
|
Stefano Emron - Generative Modeling via Denoising
(
Invited talk and Q&A
)
>
SlidesLive Video |
Stefano Ermon 🔗 |
Fri 11:00 a.m. - 11:15 a.m.
|
Compressed Sensing with Approximate Priors via Conditional Resampling
(
Contributed Talk and Q&A
)
>
SlidesLive Video We characterize the measurement complexity of compressed sensing of signals drawn from a known prior distribution, even when the support of the prior is the entire space (rather than, say, sparse vectors). We show for Gaussian measurements and \emph{any} prior distribution on the signal, that the conditional resampling estimator achieves near-optimal recovery guarantees. Moreover, this result is robust to model mismatch, as long as the distribution estimate (e.g., from an invertible generative model) is close to the true distribution in Wasserstein distance. We implement the conditional resampling estimator for deep generative priors using Langevin dynamics, and empirically find that it produces accurate estimates with more diversity than MAP. |
Ajil Jalal 🔗 |
Fri 11:15 a.m. - 11:30 a.m.
|
Chris Metzler - Approximate Message Passing (AMP) Algorithms for Computational Imaging
(
Invited Talk and Q&A
)
>
SlidesLive Video |
Christopher Metzler 🔗 |
Fri 11:30 a.m. - 12:00 p.m.
|
Discussion
(
Discussion
)
>
link
Visit the Gather.town to discuss with speakers and other attendees. |
🔗 |
Fri 1:00 p.m. - 2:00 p.m.
|
Poster Session
(
Poster Session
)
>
link
Visit the gather.town to see the posters. |
🔗 |
Fri 2:00 p.m. - 2:30 p.m.
|
Peyman Milanfar - Denoising as Building Block Theory and Applications
(
Invited talk and Q&A
)
>
SlidesLive Video |
Peyman Milanfar 🔗 |
Fri 2:30 p.m. - 3:00 p.m.
|
Rachel Ward
(
Invited talk and Q&A
)
>
|
Rachel Ward 🔗 |
Fri 3:00 p.m. - 3:30 p.m.
|
Larry Zitnick - fastMRI
(
Invited talk and Q&A
)
>
SlidesLive Video |
Larry Zitnick 🔗 |
Fri 3:30 p.m. - 4:00 p.m.
|
Discussion
(
Discussion
)
>
link
Visit the Gather.town to discuss with speakers and other attendees. |
🔗 |