Response Renormalization for Critical Deep Equilibrium Models

Deep Equilibrium Models (DEQs) compute predictions by finding a hidden representation that remains unchanged under the model's update. Training through this equilibrium uses implicit differentiation, which requires solving an adjoint system built from the residual Jacobian. If this Jacobian is nearly singular along directions to which the loss is sensitive, small perturbations can be strongly amplified in the adjoint response, producing large and highly sensitive gradients that can make optimization unreliable. We introduce Response Renormalization, a backward-pass framework that lifts selected near-pole denominators while leaving unlifted response channels unchanged. Collective Mode Response Renormalization (CMR) applies this correction in a low-dimensional critical subspace, while Phi-adaptive CMR computes a bounded response mass from a prescribed positive susceptibility rule. We derive dense and matrix-free collective formulations, distinguish exact gradients of a modified frozen-anchor residual from backward-response surrogates, and extend the construction to Structured Implicit Layers and Vector Attractors (SILVA). Across 23 multiphysics families spanning partial differential equations, three-dimensional fields, operator maps, complex geometries, and particle systems, models trained with CMR and Phi-CMR have test errors no more than five percent higher than those trained with exact implicit differentiation in more than 98% of static and 95% of transient family-seed comparisons. Solver-index experiments additionally show finite responses converging toward the static adjoint, while physical-time rollouts retain predictive fidelity under the evaluated conditions. These results demonstrate that selective response renormalization can control near-critical adjoint amplification without globally damping well-conditioned sensitivity. Therefore, the method can make parameter updates more reliable while preserving the useful gradient information needed for learning.