RA-CFGCache: From Branch-Level Criteria to Guided-Risk Control under Classifier-Free Guidance
Abstract
Diffusion models dominate visual generation, but iterative denoising remains computationally demanding, especially under Classifier-Free Guidance (CFG), which is crucial for high-fidelity generation yet nearly doubles per-step computation. To reduce this burden, recent training-free caching methods reuse intermediate predictions during sampling, yet their single-prediction criteria are misaligned with CFG sampling, where the denoising update is governed by the guided prediction rather than either branch alone. Consequently, such reuse may misestimate the error that actually perturbs sampling, leading to a branch-guided mismatch. Moreover, due to error propagation across timesteps, a local guided error may not faithfully reflect the final deviation, inducing a local-final mismatch. To address these two mismatches, we propose RA-CFGCache, a Risk-Aligned Caching framework for CFG. RA-CFGCache introduces CFG-aware Guided-Risk Composition to align caching decisions directly with the guided prediction, and Propagation-Aware Rescaling to calibrate local guided risks against the final generative deviation. Extensive experiments on FLUX.1-dev, Wan2.1-T2V-1.3B, and CogVideoX-2B show that RA-CFGCache delivers superior efficiency-fidelity trade-offs over existing training-free caching methods. Moreover, RA-CFGCache is compatible with diverse caching strategies and can further enhance their fidelity, e.g., improving PSNR over MagCache from 21.46 to 24.02 under CFG. The code is available at https://anonymous.4open.science/r/RA-CFGCache/.