Fairness Evaluation for Diffusion and Flow Models Should Account for the Generation Path: Implications for Clinical Imaging
Abstract
Diffusion and flow models are usually evaluated at their final outputs, even though each sample is produced through a sequence of intermediate states. In many applications, this trajectory is no longer an incidental part of generation: sampling may be shortened for deployment, intermediate states may be reused for editing or distillation, and downstream measurements may depend on how information is formed and preserved during generation. These considerations are especially important in medical imaging, where small changes in anatomical structure can affect quantitative biomarkers or downstream decisions even when the final images appear similar. We therefore argue that fairness evaluation for diffusion and flow models should account for the generation path whenever that path can influence the deployed system or its downstream use. We refer to this evaluation problem as counterfactual transport fairness and consider endpoint measures together with time-resolved differences in subgroup distributions, movement, and task-relevant content, while accounting for variables that may legitimately affect generation. A simple transport argument shows that controlling differences along the path can constrain endpoint discrepancy, whereas endpoint agreement alone does not reveal how that endpoint was reached. We illustrate this gap using a neural Schr\"odinger-bridge model for fundus-to-OCTA translation, where saved states reveal weak BMI-subgroup recoverability and localized movement differences that are not apparent from endpoint summaries. These observations motivate path-aware evaluation as a complement to endpoint auditing and identify several questions for future work on deployment robustness and clinically meaningful generative-model evaluation.