Beyond One Reference: A Controlled Stress Test of NIR-to-RGB Evaluation under Spectral Ambiguity

NIR-to-RGB translation is commonly evaluated against one RGB reference per NIR input even though the inverse mapping may admit multiple plausible visible outputs. We investigate whether this single-reference protocol can select a different model family from a proper repeated-target distributional evaluation. SPECTRASIM deliberately fixes a rendered NIR observation while resampling eight compatible RGB targets; its multimodality is controlled by design and is not claimed as a calibrated physical-camera model. Under a common data split, shared U-Net trunk, optimizer, training schedule, and 600-update budget, a $K=4$ winner-take-all (WTA) predictor reduces energy score from $0.428\pm0.001$ to $0.319\pm0.003$ relative to $\ell_2$ regression while changing matched-baseline PSNR by only $-0.30$ dB. A distinct $K=4$ mixture density network reaches $0.322\pm0.001$, showing that the ranking change is not specific to WTA. WTA also achieves a $0.50$ dB oracle-of-four PSNR gain over its MAP prediction, indicating that alternative hypotheses cover held-out targets. Finally, a frozen RGB-to-NIR consistency objective, evaluated using a separately trained forward model, reveals a trade-off between forward consistency and distributional fit. The results demonstrate that point fidelity, predictive-distribution quality, and forward consistency can rank NIR-to-RGB predictors differently. We make a controlled evaluation claim only; real-sensor spectral validation remains necessary before drawing conclusions about physical NIR ambiguity.