XMNoise2Clean: Cross-Modal Denoising Under Sparse Data

Self-supervised image denoisers such as Blind2Unblind excel when clean targets are unavailable but noisy observations are plentiful. We ask: can widely available source-domain data, such as RGB images, improve these denoisers when noisy observations are also limited? We introduce XMNoise2Clean, a framework that learns a noisy target generator from limited paired source/target data, applies it to abundant source images, and trains the denoiser on the union of real and generated noisy samples. We define a cross-modal predictability metric Γ ∈ [0, 1] that measures what fraction of target signal variance the source can explain. When Γ ≈ 1 (no scene properties hidden from the source), generated samples are in-domain and help substantially: using a generator calibrated on 500 paired examples, XMNoise2Clean improves Blind2Unblind by about 3.6 dB with only 50 downstream denoiser-stage real target images, nearly matching the 1000-real denoiser baseline. When Γ is moderate (one hidden scene property, such as sub-frame temporal dynamics in event cameras), the method helps but less reliably. When Γ is low (two or more hidden scene properties, such as the temperature and emissivity fields that determine thermal infrared intensity but are invisible to RGB), generated samples hardly help and can suppress real target structure. Controlled synthetic experiments with known Γ confirm that augmentation benefit generally increases with source-target predictability, and three real-world modality settings plus a controlled proxy experiment support the pattern.