Workshop
NeurIPS 2023 Workshop on Diffusion Models
Bahjat Kawar 路 Valentin De Bortoli 路 Charlotte Bunne 路 James Thornton 路 Jiaming Song 路 Jong Chul Ye 路 Chenlin Meng
Hall B1 (level 1)
Fri 15 Dec, 6:50 a.m. PST
Over the past three years, diffusion models have established themselves as a new generative modeling paradigm. Their empirical successes have broadened the applications of generative modeling to image, video, audio, 3D synthesis, science applications, and more. As diffusion models become more and more popular and are applied to extremely diverse problems, it also becomes harder to follow the key contributions in the field. This workshop aims to keep track of recent advances and identify guidelines for future research. By bringing together practice, methodology, and theory actors we aim to identify unexplored areas, foster collaboration, and push the frontier of diffusion model research.
Link to website: https://diffusionworkshop.github.io/
Ask questions to our panelists here: https://docs.google.com/forms/d/e/1FAIpQLSeTRsWFvKlsFg31K8Vq6hHGOydmvd7YNMuOLOCcKgqSqO8mXw/viewform
Schedule
Fri 6:50 a.m. - 7:00 a.m.
|
Opening Remarks
(
Opening Remarks
)
>
|
馃敆 |
Fri 7:00 a.m. - 7:30 a.m.
|
Tali Dekel: Pretrained diffusion is all we need: a journey beyond training distribution
(
Keynote
)
>
SlidesLive Video |
馃敆 |
Fri 7:30 a.m. - 8:00 a.m.
|
Brian Trippe & Jason Yim: De novo design of protein structure and function with RFdiffusion
(
Keynote
)
>
SlidesLive Video |
馃敆 |
Fri 8:00 a.m. - 8:15 a.m.
|
Coffee Break
|
馃敆 |
Fri 8:15 a.m. - 9:15 a.m.
|
Poster Session 1
(
Poster session
)
>
|
馃敆 |
Fri 9:15 a.m. - 10:00 a.m.
|
Panel Discussion: Arash Vahdat, Ruiqi Gao, Tim Salimans, Robin Rombach
(
Panel Discussion
)
>
SlidesLive Video |
馃敆 |
Fri 10:00 a.m. - 11:30 a.m.
|
Lunch
|
馃敆 |
Fri 11:30 a.m. - 12:00 p.m.
|
Sayak Paul: Controlling Text-to-Image Diffusion Models
(
Keynote
)
>
SlidesLive Video |
馃敆 |
Fri 12:00 p.m. - 12:30 p.m.
|
Yang Song: Consistency Models
(
Keynote
)
>
SlidesLive Video |
馃敆 |
Fri 12:30 p.m. - 12:40 p.m.
|
LC-SD: Realistic Endoscopic Image Generation with Stable Diffusion and ControlNet
(
Oral
)
>
link
SlidesLive Video |
Joanna Kaleta 路 Diego Dall'Alba 路 Szymon Plotka 路 Przemyslaw Korzeniowski 馃敆 |
Fri 12:40 p.m. - 12:50 p.m.
|
Manifold Diffusion Fields
(
Oral
)
>
link
SlidesLive Video |
Ahmed Elhag 路 Yuyang Wang 路 Joshua Susskind 路 Miguel Angel Bautista 馃敆 |
Fri 12:50 p.m. - 1:00 p.m.
|
The Emergence of Reproducibility and Consistency in Diffusion Models
(
Oral
)
>
link
SlidesLive Video |
Huijie Zhang 路 Jinfan Zhou 路 Yifu Lu 路 Minzhe Guo 路 Liyue Shen 路 Qing Qu 馃敆 |
Fri 1:00 p.m. - 1:15 p.m.
|
Coffee Break
|
馃敆 |
Fri 1:15 p.m. - 2:15 p.m.
|
Invited Lightning Talks
(
Lightning Talks
)
>
SlidesLive Video |
馃敆 |
Fri 2:15 p.m. - 3:15 p.m.
|
Poster Session 2
(
Poster Session
)
>
|
馃敆 |
Fri 3:15 p.m. - 3:30 p.m.
|
Closing Remarks + Award Reveal
(
Closing Remarks
)
>
SlidesLive Video |
馃敆 |
-
|
TS-DiffuGen: An equivariant diffusion model for reaction transition state conformation generation ( Poster ) > link | Sacha Raffaud 路 Jeff Guo 路 Philippe Schwaller 馃敆 |
-
|
EDGE++: Improved Training and Sampling of EDGE ( Poster ) > link | Xiaohui Chen 路 Mingyang Wu 路 Liping Liu 馃敆 |
-
|
The Hidden Linear Structure in Score-Based Models and its Application ( Poster ) > link | Binxu Wang 路 John Vastola 馃敆 |
-
|
Controllable Music Production with Diffusion Models and Guidance Gradients ( Poster ) > link | Mark Levy 路 Bruno Di Giorgi 路 Floris Weers 路 Angelos Katharopoulos 路 Tom R Nickson 馃敆 |
-
|
Effective Quantization for Diffusion Models on CPUs ( Poster ) > link | Hanwen Chang 路 Haihao Shen 路 Yiyang Cai 路 Xinyu Ye 路 Zhenzhong Xu 路 Wenhua Cheng 路 Weiwei Zhang 路 Yintong Lu 路 Heng Guo 馃敆 |
-
|
Staged Diffusion Models with Analytically Designed Hyperparameters ( Poster ) > link | Kanad Pardeshi 路 Shrey Singla 路 Sunita Sarawagi 馃敆 |
-
|
VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing ( Poster ) > link | Paul Couairon 路 Cl茅ment Rambour 路 Jean-Emmanuel HAUGEARD 路 Nicolas THOME 馃敆 |
-
|
Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion ( Poster ) > link | Dongjun Kim 路 Chieh-Hsin Lai 路 Wei-Hsiang Liao 路 Naoki Murata 路 Yuhta Takida 路 Toshimitsu Uesaka 路 Yutong He 路 Yuki Mitsufuji 路 Stefano Ermon 馃敆 |
-
|
Learning from Invalid Data: On Constraint Satisfaction in Generative Models ( Poster ) > link | Giorgio Giannone 路 Lyle Regenwetter 路 Akash Srivastava 路 Dan Gutfreund 路 Faez Ahmed 馃敆 |
-
|
Diffusion-C: Unveiling the Generative Challenges of Diffusion Models through Corrupted Data ( Poster ) > link | Keywoong Bae 路 Suan Lee 路 Wookey Lee 馃敆 |
-
|
Beyond Generation: Exploring Generalization of Diffusion Models in Few-shot Segmentation ( Poster ) > link | Jie Liu 路 TAO HU 路 Jan-jakob Sonke 路 Efstratios Gavves 馃敆 |
-
|
Improving Discrete Diffusion Models via Structured Preferential Generation ( Poster ) > link | Severi Rissanen 路 Markus Heinonen 路 Arno Solin 馃敆 |
-
|
Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization ( Poster ) > link | Yinbin Han 路 Meisam Razaviyayn 路 Renyuan Xu 馃敆 |
-
|
Decoding the Seeing Brain: Reconstructing Images and Text from fMRI Recordings with multimodal diffusion models ( Poster ) > link | Matteo Ferrante 路 Tommaso Boccato 路 Furkan Ozcelik 路 Rufin VanRullen 路 Nicola Toschi 馃敆 |
-
|
Fast Sampling via De-randomization for Discrete Diffusion Models ( Poster ) > link | Zixiang Chen 路 Angela Yuan 路 Yongqian Li 路 Yiwen Kou 路 Junkai Zhang 路 Quanquan Gu 馃敆 |
-
|
Diffusion models for probabilistic programming ( Poster ) > link | Simon Dirmeier 路 Fernando Perez-Cruz 馃敆 |
-
|
A Denoising Diffusion Model for Synthetic Fluid Field Prediction ( Poster ) > link | Gefan Yang 路 Stefan Sommer 馃敆 |
-
|
AlphaFold Meets Flow Matching for Generating Protein Ensembles ( Poster ) > link | Bowen Jing 路 Bonnie Berger 路 Tommi Jaakkola 馃敆 |
-
|
SyncDiffusion: Coherent Montage via Synchronized Joint Diffusions ( Poster ) > link | Yuseung Lee 路 Kunho Kim 路 Hyunjin Kim 路 Minhyuk Sung 馃敆 |
-
|
Retrofitting 2D Latent Diffusion for Controllable Face Image Generation ( Poster ) > link | Weihao Xia 路 Cengiz Oztireli 路 Jing-Hao Xue 馃敆 |
-
|
IterInv: Iterative Inversion for Pixel-Level T2I Models ( Poster ) > link | Chuanming Tang 路 kai wang 路 Joost van de Weijer 馃敆 |
-
|
Investigating the Adversarial Robustness of Density Estimation Using the Probability Flow ODE ( Poster ) > link | Marius Arvinte 路 Cory Cornelius 路 Jason Martin 路 Nageen Himayat 馃敆 |
-
|
Towards Fast Stochastic Sampling in Diffusion Generative Models ( Poster ) > link | Kushagra Pandey 路 Maja Rudolph 路 Stephan Mandt 馃敆 |
-
|
Motion Flow Matching for Efficient Human Motion Synthesis and Editing ( Poster ) > link | TAO HU 路 Wenzhe Yin 路 Pingchuan Ma 路 Yunlu Chen 路 Basura Fernando 路 Yuki M Asano 路 Efstratios Gavves 路 Pascal Mettes 路 Bj枚rn Ommer 路 Cees Snoek 馃敆 |
-
|
In search of dispersed memories: Generative diffusion models are associative memory networks ( Poster ) > link | Luca Ambrogioni 馃敆 |
-
|
Improved Convergence of Score-Based Diffusion Models via Prediction-Correction ( Poster ) > link | Francesco Pedrotti 路 Jan Maas 路 Marco Mondelli 馃敆 |
-
|
You Only Submit One Image to Find the Most Suitable Generative Model ( Poster ) > link | Zhi Zhou 路 Lan-Zhe Guo 路 Pengxiao Song 路 Yu-Feng Li 馃敆 |
-
|
Denoising Heat-inspired Diffusion with Insulators for Collision Free Motion Planning ( Poster ) > link | Junwoo Chang 路 Hyunwoo Ryu 路 Jiwoo Kim 路 Soochul Yoo 路 Joohwan Seo 路 Nikhil Potu Surya Prakash 路 Jongeun Choi 路 Roberto Horowitz 馃敆 |
-
|
DiffEnc: Variational Diffusion with a Learned Encoder ( Poster ) > link | Beatrix M. G. Nielsen 路 Anders Christensen 路 Andrea Dittadi 路 Ole Winther 馃敆 |
-
|
Beyond U: Making Diffusion Models Faster & Lighter ( Poster ) > link | Sergio Calvo Ordo帽ez 路 Jiahao Huang 路 Lipei Zhang 路 Guang Yang 路 Carola-Bibiane Sch枚nlieb 路 ANGELICA I AVILES-RIVERO 馃敆 |
-
|
Understanding Denoising Diffusion Probabilistic Models and their Noise Schedules via the Ornstein--Uhlenbeck Process ( Poster ) > link | Javier E. Santos 路 Yen Ting Lin 馃敆 |
-
|
Diffusion based Zero-shot Medical Image-to-Image Translation for Cross Modality Segmentation ( Poster ) > link | zihao wang 路 YINGYU YANG 路 Yuzhou Chen 路 Tingting Yuan 路 Maxime Sermesant 路 Herve Delingette 路 Ona Wu 馃敆 |
-
|
Harmonic Prior Self-conditioned Flow Matching for Multi-Ligand Docking and Binding Site Design ( Poster ) > link | Hannes St盲rk 路 Bowen Jing 路 Regina Barzilay 路 Tommi Jaakkola 馃敆 |
-
|
Sharp analysis of learning a flow-based generative model from limited sample complexity ( Poster ) > link | Hugo Cui 路 Eric Vanden-Eijnden 路 Florent Krzakala 路 Lenka Zdeborov谩 馃敆 |
-
|
Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective ( Poster ) > link | Davide Ghio 路 Yatin Dandi 路 Florent Krzakala 路 Lenka Zdeborov谩 馃敆 |
-
|
Enhancing Diffusion-based Point Cloud Generation with Smoothness Constraint ( Poster ) > link | Yukun Li 路 Liping Liu 馃敆 |
-
|
Circumventing Concept Erasure Methods For Text-to-Image Generative Models ( Poster ) > link | Minh Pham 路 Kelly Marshall 路 Niv Cohen 路 Govind Mittal 路 Chinmay Hegde 馃敆 |
-
|
Latent Diffusion for Document Generation with Sequential Decoding ( Poster ) > link | Zihuiwen Ye 路 Elle Michelle Yang 路 Phil Blunsom 馃敆 |
-
|
The Emergence of Reproducibility and Consistency in Diffusion Models ( Poster ) > link | Huijie Zhang 路 Jinfan Zhou 路 Yifu Lu 路 Minzhe Guo 路 Liyue Shen 路 Qing Qu 馃敆 |
-
|
Exploring Attribute Variations in Style-based GANs using Diffusion Models ( Poster ) > link | Rishubh Parihar 路 Prasanna Balaji 路 Raghav Magazine 路 Sarthak Vora 路 Tejan Karmali 路 Varun Jampani 路 Venkatesh Babu R 馃敆 |
-
|
LDM3D-VR: Latent Diffusion Model for 3D VR ( Poster ) > link | Gabriela Ben-Melech Stan 路 Diana Wofk 路 Estelle Aflalo 路 Shao-Yen Tseng 路 zhipeng cai 路 Michael Paulitsch 路 VASUDEV LAL 馃敆 |
-
|
DiffDock-Pocket: Diffusion for Pocket-Level Docking with Sidechain Flexibility ( Poster ) > link | Michael Plainer 路 Marcella Toth 路 Simon Dobers 路 Hannes St盲rk 路 Gabriele Corso 路 C茅line Marquet 路 Regina Barzilay 馃敆 |
-
|
Probing Intersectional Biases in Vision-Language Models with Counterfactual Examples ( Poster ) > link | Phillip Howard 路 Avinash Madasu 路 Tiep Le 路 Gustavo Lujan-Moreno 路 VASUDEV LAL 馃敆 |
-
|
WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Models ( Poster ) > link | Changhoon Kim 路 Kyle Min 路 Maitreya Patel 路 Sheng Cheng 路 'YZ' Yezhou Yang 馃敆 |
-
|
-GANs Settle Scores! ( Poster ) > link | Siddarth Asokan 路 Nishanth Shetty 路 Aadithya Srikanth 路 Chandra Seelamantula 馃敆 |
-
|
Latent Painter ( Poster ) > link | Shih-Chieh Su 馃敆 |
-
|
LoRA can Replace Time and Class Embeddings in Diffusion Probabilistic Models ( Poster ) > link | Joo Young Choi 路 Jaesung Park 路 Inkyu Park 路 Jaewoong Cho 路 Albert No 路 Ernest Ryu 馃敆 |
-
|
Importance-Guided Diffusion ( Poster ) > link | Paris Flood 路 Pietro Li贸 馃敆 |
-
|
Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance ( Poster ) > link | Niket Patel 路 Luis Salamanca 路 Luis Barba 馃敆 |
-
|
Particle Guidance: non-I.I.D. Diverse Sampling with Diffusion Models ( Poster ) > link | Gabriele Corso 路 Yilun Xu 路 Valentin De Bortoli 路 Regina Barzilay 路 Tommi Jaakkola 馃敆 |
-
|
Discrete Diffusion Language Modeling by Estimating the Ratios of the Data Distribution ( Poster ) > link | Aaron Lou 路 Chenlin Meng 路 Stefano Ermon 馃敆 |
-
|
Adversarial Estimation of Topological Dimension with Harmonic Score Maps ( Poster ) > link | Eric Yeats 路 Cameron Darwin 路 Frank Liu 路 Hai Li 馃敆 |
-
|
Generative Fractional Diffusion Models ( Poster ) > link |
11 presentersGabriel Nobis 路 Marco Aversa 路 Maximilian Springenberg 路 Michael Detzel 路 Stefano Ermon 路 Shinichi Nakajima 路 Roderick Murray-Smith 路 Sebastian Lapuschkin 路 Christoph Knochenhauer 路 Luis Oala 路 Wojciech Samek |
-
|
Text-Aware Diffusion Policies ( Poster ) > link | Calvin Luo 路 Chen Sun 馃敆 |
-
|
CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images ( Poster ) > link | Aaron Gokaslan 路 A. Feder Cooper 路 Jasmine Collins 路 Landan Seguin 路 Austin Jacobson 路 Mihir Patel 路 Jonathan Frankle 路 Cory Stephenson 路 Volodymyr Kuleshov 馃敆 |
-
|
Beyond Surface Statistics: Scene Representations in a Latent Diffusion Model ( Poster ) > link | Yida Chen 路 Fernanda Vi茅gas 路 Martin Wattenberg 馃敆 |
-
|
DiffusionShield: A Watermark for Data Copyright Protection against Generative Diffusion Models ( Poster ) > link | Yingqian Cui 路 Jie Ren 路 Han Xu 路 Pengfei He 路 Hui Liu 路 Lichao Sun 路 Yue XING 路 Jiliang Tang 馃敆 |
-
|
Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models ( Poster ) > link | Song Mei 路 Yuchen Wu 馃敆 |
-
|
Drag-guided diffusion models for vehicle image generation ( Poster ) > link | Nikos Arechiga 路 Frank Permenter 路 Binyang Song 路 Chenyang Yuan 馃敆 |
-
|
Generate Your Own Scotland: Satellite Image Generation Conditioned on Maps ( Poster ) > link | Miguel Espinosa Minano 路 Elliot Crowley 馃敆 |
-
|
DFU: scale-robust diffusion model for zero-shot super-resolution image generation ( Poster ) > link | Alexander Havrilla 路 Kevin Rojas 路 Wenjing Liao 路 Molei Tao 馃敆 |
-
|
Diffusion Models without Attention ( Poster ) > link | Nathan Yan 路 Jiatao Gu 路 Alexander Rush 馃敆 |
-
|
Effective Data Augmentation With Diffusion Models ( Poster ) > link | Brandon Trabucco 路 Kyle Doherty 路 Max Gurinas 路 Russ Salakhutdinov 馃敆 |
-
|
DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting ( Poster ) > link | Salva R眉hling Cachay 路 Bo Zhao 路 Hailey Joren 路 Rose Yu 馃敆 |
-
|
ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models ( Poster ) > link | Maitreya Patel 路 Tejas Gokhale 路 Chitta Baral 路 'YZ' Yezhou Yang 馃敆 |
-
|
Functional Flow Matching ( Poster ) > link | Gavin Kerrigan 路 Giosue Migliorini 路 Padhraic Smyth 馃敆 |
-
|
Diffusion-Augmented Neural Processes ( Poster ) > link | Lorenzo Bonito 路 James Requeima 路 Aliaksandra Shysheya 路 Richard Turner 馃敆 |
-
|
Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks ( Poster ) > link | Luca Scimeca 路 Alexander Rubinstein 路 Armand Nicolicioiu 路 Damien Teney 路 Yoshua Bengio 馃敆 |
-
|
Score Normalization for a Faster Diffusion Exponential Integrator Sampler (DEIS) ( Poster ) > link | Guoxuan Xia 路 Duolikun Danier 路 Ayan Das 路 Stathi Fotiadis 路 Farhang Nabiei 路 Ushnish Sengupta 路 Alberto Bernacchia 馃敆 |
-
|
AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Model ( Poster ) > link | Zibin Dong 路 Yifu Yuan 路 Jianye Hao 路 Fei Ni 路 Yao Mu 路 YAN ZHENG 路 Yujing Hu 路 Tangjie Lv 路 Changjie Fan 路 ZHIPENG HU 馃敆 |
-
|
LC-SD: Realistic Endoscopic Image Generation with Stable Diffusion and ControlNet ( Poster ) > link | Joanna Kaleta 路 Diego Dall'Alba 路 Szymon Plotka 路 Przemyslaw Korzeniowski 馃敆 |
-
|
Faster Training of Diffusion Models and Improved Density Estimation via Parallel Score Matching ( Poster ) > link | Etrit Haxholli 路 Marco Lorenzi 馃敆 |
-
|
Masked Multi-time Diffusion for Multi-modal Generative Modeling ( Poster ) > link | Mustapha BOUNOUA 路 Giulio Franzese 路 Pietro Michiardi 馃敆 |
-
|
MOFDiff: Coarse-grained Diffusion for Metal-Organic Framework Design ( Poster ) > link | Xiang Fu 路 Tian Xie 路 Andrew Rosen 路 Tommi Jaakkola 路 Jake Smith 馃敆 |
-
|
Successfully Applying Lottery Ticket Hypothesis to Diffusion Model ( Poster ) > link | Chao Jiang 路 Bo Hui 路 Bohan Liu 路 Da Yan 馃敆 |
-
|
Strong generalization in diffusion models ( Poster ) > link | Zahra Kadkhodaie 路 Florentin Guth 路 Eero Simoncelli 路 Stephane Mallat 馃敆 |
-
|
Diffusion Models With Learned Adaptive Noise Processes ( Poster ) > link | Subham Sahoo 路 Aaron Gokaslan 路 Christopher De Sa 路 Volodymyr Kuleshov 馃敆 |
-
|
Generalized Contrastive Divergence: Joint Training of Energy-Based Model and Diffusion Model through Inverse Reinforcement Learning ( Poster ) > link | Sangwoong Yoon 路 Dohyun Kwon 路 Himchan Hwang 路 Yung-Kyun Noh 路 Frank Park 馃敆 |
-
|
Diffusing More Objects for Semi-Supervised Domain Adaptation with Less Labeling ( Poster ) > link | Leander van den Heuvel 路 Gertjan Burghouts 路 David Zhang 路 Gwenn Englebienne 路 Sabina van Rooij 馃敆 |
-
|
Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generation ( Poster ) > link | Giorgio Giannone 路 Akash Srivastava 路 Ole Winther 路 Faez Ahmed 馃敆 |
-
|
TADA: Timestep-Aware Data Augmentation for Diffusion Models ( Poster ) > link | NaHyeon Park 路 Kunhee Kim 路 Song Park 路 Jung-Woo Ha 路 Hyunjung Shim 馃敆 |
-
|
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints ( Poster ) > link | Jian Chen 路 Ruiyi Zhang 路 Yufan Zhou 路 Rajiv Jain 路 Zhiqiang Xu 路 Ryan Rossi 路 Changyou Chen 馃敆 |
-
|
Manifold Diffusion Fields ( Poster ) > link | Ahmed Elhag 路 Yuyang Wang 路 Joshua Susskind 路 Miguel Angel Bautista 馃敆 |
-
|
Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models ( Poster ) > link | Hyeonho Jeong 路 Jong Chul Ye 馃敆 |