CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation

Diffusion models have emerged as powerful tools for generative tasks, producing high-quality outputs across diverse domains. However, how the generated data responds to the initial noise perturbation in diffusion models remains under-explored, hindering a deeper understanding of the controllability of the sampling process. In this work, we first observe an interesting phenomenon: the relationship between the change of generation outputs and the scale of initial noise perturbation is highly linear through the diffusion ODE sampling process. We then provide both theoretical and empirical analyses to justify this linearity property of the input–output (noise → generation data) relationship. Inspired by these insights, we propose a novel Controllable and Constrained Sampling (CCS) method, along with a new controller algorithm for diffusion models, that enables precise control over both (1) the proximity of individual samples to a target image and (2) the alignment of the sample mean with the target, while preserving high sample quality. We conduct extensive experiments comparing our proposed sampling approach with other methods in terms of both sampling controllability and generated data quality. Results show that CCS achieves significantly more precise controllability while maintaining superior sample quality and diversity, enabling practical applications such as fine-grained and robust image editing. Code: https://github.com/efzero/diffusioncontroller