Reversing the Roles of Signal and Noise: Modeling Noise in Astronomical Images without Paired Data

Many scientific instruments produce structured noise that contaminates the signal of interest and cannot be observed in isolation, ruling out standard supervised denoising. We propose BEAM (Background Estimation via Additive Modeling), a generative framework for modeling complex noise in these settings. As ground truth noise observations are unavailable, BEAM learns a generative model of the clean signal and uses it to score candidate noise estimates by how plausible the underlying signal looks. Score matching on signal data turns this into a tractable likelihood that we sample from with Langevin dynamics, which refines a coarse classical estimate of the noise into a more accurate one. We further show that when metadata governing the noise is known in advance, a separate conditional flow model predicts the noise from that metadata alone, enabling pre-observation forecasting of the noise. We instantiate BEAM on stray light in NASA's Transiting Exoplanet Survey Satellite (TESS), where contamination from the Earth and Moon obscures stellar signals. BEAM produces cleaner stray light estimates than the filtering tool used in current TESS pipelines and forecasts stray-light contamination across the observing window for the interstellar comet 3I/ATLAS using only spacecraft--Earth--Moon geometry, supporting prospective scheduling of time-critical observations and a path to recovering faint signals that current pipelines discard.