A Sampling-Based Domain Generalization Study with Diffusion Generative Models

In this work, we investigate the domain generalization capabilities of diffusion models, specifically in the context of an application scenario where the task objective is to synthesize images distinct from the training data. Unlike popular fine-tuning approaches, we propose investigating and tackling this challenge from a sampling-based perspective using frozen, pre-trained diffusion models. Specifically, we begin by revealing that diffusion models trained on single-domain images are already equipped with sufficient representation abilities to reconstruct arbitrary out-of-domain (OOD) images from the inverted latent encoding following bi-directional deterministic diffusion and denoising trajectories. In addition, such OOD images establish Gaussian priors in the latent spaces of the given model, and the inverted latent modes are separable from their initial training domain. The above OOD latent properties allow us to synthesize new images of the target unseen domain by discovering qualified OOD latent encodings in the inverted noisy spaces, which is fundamentally different from the current paradigm that seeks to modify the denoising trajectory to achieve the same goal by tuning the model parameters. Our cross-model and domain experiments show that the proposed sampling-based method can expand the latent space and generate unseen images via frozen diffusion models without impairing the quality of generation of their original domain. We also showcase a practical application of our approach in dramatically different domains using astrophysical data, revealing the potential of such a generalization paradigm in data-sparse fields such as scientific explorations.