Scale Is Not Time: Train the Graph, Not the Schedule
Mayank Singal
Abstract
Progressive-resolution diffusion saves compute by denoising at low resolution before moving to higher resolution. This makes the resolution schedule look like an inference-time choice, yet moving a transition can ruin generation even when the new path is valid and has almost the same cost. We trace this brittleness to fixed-schedule training, which exposes each diffusion time at only one resolution. The model learns the native scale-time path, not the surrounding process. We distinguish denoising at an off-path state from changing resolution at an unfamiliar time and find that the two capabilities are not interchangeable. With matched training budgets, node-only adaptation leaves an across-path KID spread of $143\pm10$, while transition exposure reduces it to $2.8\pm1.3$. Training both states and transitions produces schedule-elastic samplers on CelebA and ImageNet, including useful cheaper and equal-compute trajectories. This lets the resolution schedule act as a deployment policy rather than a fixed part of training.
Chat is not available.
Successful Page Load