A Catalog-Free Spatio-Spectral Representation of the Sky

Astronomical surveys observe the same sky through instruments with different bandpasses, point-spread functions, apertures, and noise properties, and standard coaddition offers no way to combine a fiber spectrum with a broadband image. We model the sky as a single continuous spectral surface-brightness field, fit directly to calibrated single-epoch measurements, each modeled as an integral of the field against its instrument's own measurement kernel (astrometry, PSF or line-spread function, bandpass or wavelength bin, and aperture). An implementation with $9.2\times10^6$ anisotropic Gaussian components and 32 shared nonnegative spectral templates jointly fits 405 SDSS exposures and 889 SDSS/BOSS fiber spectra over an $84'\times84'$ COSMOS field without an external source catalog. The in-sample forward model recovers integrated source flux in these fibers, with the known sky subtracted, to 5.5\% median absolute fractional error. We then omit complete exposures from optimization, initialization, and component lifecycle, retain other bands and spectra as same-sky conditioning, and synthesize the omitted frames through their recorded operators. In four holdouts with more than 70\% of the footprint lacking same-band coverage, hole-region flux ratios range from 0.71 to 1.86, versus 0.90--1.06 for matched in-sample controls. Coverage stratification separates supported from unsupported synthesis and localizes the failures. The result is both a scalable shared representation and a falsifiable protocol for testing observation-operator transfer.