Optimizer or Precision? Understanding Optimizer–Precision Interactions in PINNs

Physics-informed neural networks (PINNs) have been shown to reach low training loss without recovering an accurate solution, and recent work has attributed some of these failures to insufficient arithmetic precision. Those studies evaluate precision under L-BFGS, leaving open whether the observed sensitivity is general to low-precision training or specific to the optimizer. In this work, we separate the two factors across eight configurations on four PDE benchmarks, varying precision, optimizer, and scheduled switches between them. Across 32 runs, Adam-terminating configurations remain substantially less accurate, while every run reaching low solution error terminates in L-BFGS. Among static runs, doubling precision changes Adam error by at most $1.25\times$, compared with as much as $17.7\times$ under L-BFGS. At fixed FP64, changing from Adam to L-BFGS lowers error on all four benchmarks. Under the equal-weight main setup, beginning in FP32 Adam and switching to FP64 L-BFGS improves on full FP64 L-BFGS on three of four benchmarks. Our results show that arithmetic precision and optimizer choice interact strongly in PINN training, with the largest precision effects arising under L-BFGS.