Denoisers with Differentiable Constraint Knowledge for Diffusion Models

Diffusion models are powerful generative models for complex data distributions, yet they often struggle to generate samples that precisely satisfy constraints inherent in scientific applications. While recent approaches have introduced regularization terms or guidance methods to enforce such constraints, they lead to bias in the generative distribution, compromising the model's ability to faithfully represent the true data distribution. In this extended abstract, we propose a different approach that embeds arbitrary denoiser architectures with differentiable constraints as inductive biases from initialization, maintaining the asymptotic unbiasedness of standard denoising score matching. Through experiments on a representative PDE problem, we show that our method generates constraint-compliant samples without the distributional biases introduced by current methods.