Practical and Asymptotically Exact Conditional Sampling in Diffusion Models

Luhuan Wu · Brian Trippe · Christian Naesseth · David Blei · John Cunningham

Great Hall & Hall B1+B2 (level 1) #1226
[ ]
Thu 14 Dec 8:45 a.m. PST — 10:45 a.m. PST


Diffusion models have been successful on a range of conditional generation tasks including molecular design and text-to-image generation. However, these achievements have primarily depended on task-specific conditional training or error-prone heuristic approximations. Ideally, a conditional generation method should provide exact samples for a broad range of conditional distributions without requiring taskspecific training. To this end, we introduce the Twisted Diffusion Sampler, or TDS. TDS is a sequential Monte Carlo (SMC) algorithm that targets the conditional distributions of diffusion models. The main idea is to use twisting, an SMC technique that enjoys good computational efficiency, to incorporate heuristic approximations without compromising asymptotic exactness. We first find in simulation and in conditional image generation tasks that TDS provides a computational statistical trade-off, yielding more accurate approximations with many particles but with empirical improvements over heuristics with as few as two particles. We then turn to motif-scaffolding, a core task in protein design, using a TDS extension to Riemannian diffusion models; on benchmark test cases, TDS allows flexible conditioning criteria and often outperforms the state-of-the-art, conditionally trained model.

Chat is not available.