Uniform Diffusion Models revisited: Leave-One-Out Denoiser and Absorbing State Reformulation

Discrete diffusion models are typically trained by predicting the clean data given a noisy observation, which is then used with a certain choice of reverse transition parameterization. Two choices coexist in the literature, which we refer to as marginalization and plug-in. Although they coincide in the case of masked diffusion models (MDMs), we revisit this question for uniform diffusion models (UDMs), where the distinction matters. We show that the plug-in parameterization is not optimized by the denoising posterior but by a \emph{leave-one-out} posterior that removes the influence of the local observation. We give a thorough characterization of the resulting training target, together with exact conversion formulas between the denoiser, the \emph{leave-one-out} posterior, and the score. As a by-product, the leave-one-out characterization yields a Gibbs-based predictor–corrector scheme at no additional training cost. We further propose an absorbing-state construction of uniform diffusion, whose denoiser decomposes into simpler absorbing denoising posteriors, and which recovers the full UDM joint law following a simple resampling step. Empirically, we show that these design choices interact strongly with optimization and generative quality; on language modeling, the \emph{leave-one-out} denoiser improves over other parameterizations.