Is Beating Adam Enough? Operator Conditioning and the Accuracy of Squared-Residual PDE Solvers

Physics-informed and spectral neural PDE solvers minimize a squared residual objective as part of their optimization approach, and a growing literature reports solutions with accuracy nearing machine precision. We show that in an important regime overlapping with real-world engineering problems this accuracy can be illusory, because it depends on the exact solution used to test the solver, and not necessarily on the solver itself. We study the indefinite Helmholtz equation near a resonance, where the discretized system is severely ill-conditioned, and test three solution regimes: (1) smooth, (2) typical (random), and (3) resonant (aligned with the mode that dominates physical responses near resonance). Second-order and preconditioned solvers do reach higher accuracy nearing machine precision for the smooth solutions used as benchmarks. However, their accuracy degrades when the test solution overlaps with the operator's worst-conditioned mode: at high condition number, the $L_2$ relative error (L2RE), which is inversely proportional to the accuracy, degrades as $\sim10^{-11}$ (smooth) $\to$ $\sim10^{-1}$ (typical) $\to$ $\sim10^{0}$ (resonant). We find that solving the un-squared least-squares system instead recovers high accuracy across all three regimes. We trace the cause to classical numerics, where squaring the residual in turn squares the condition number of the system the optimizer must descend. Furthermore, a small residual no longer certifies a small error, because error along the worst-conditioned direction barely registers in the residual. Moreover, we find that the source of ill-conditioning matters, as ill-conditioning stemming from the high-wavenumber pollution effect does not result in accuracy loss, but one stemming from approaching a resonance degrades the accuracy. Testing on a single solution aligned with this near-null mode makes the hidden error visible, reinforcing an important insight: the exact solution a solver is tested on is part of the accuracy claim itself, rather than an incidental reporting detail.