Grid or Set? Representations for Learned VLBI Reconstruction Under Distribution Shift

Radio interferometers measure sparse, irregularly sampled complex visibilities rather than images, and each observation corresponds to a different measurement operator determined by its particular $uv$-coverage. We study how the choice of representation for these irregular measurements affects learned reconstruction and its robustness to changes in sampling geometry, training parameter-matched Grid-CNN and DeepSets models on real paired Astrogeo X-band visibilities and CLEAN images, with the independently observed MOJAVE survey held out entirely as an out-of-distribution (OOD) test set. DeepSets attains lower absolute NMSE than the gridded-visibility CNN baseline at both in-distribution (ID) and OOD splits (ID: 0.228 vs.\ 0.376; OOD: 0.448 vs.\ 0.707) and wins on 85.6\% of paired OOD comparisons, yet degrades more in relative ID$\to$OOD terms (+96.4\% vs.\ +88.1\%): absolute quality and distribution-shift robustness disagree on which representation is preferable. We also find a metric-level reversal: median SSIM is higher on MOJAVE than on X-band for both models even as NMSE roughly doubles, not previously highlighted for this comparison. We also describe an architectural pitfall -- a spatially collapsed decoder input -- found and fixed in the DeepSets decoder, likely relevant to any architecture that pools a permutation-invariant representation into a convolutional decoder.