Second-Order Optimization with Asynchronous Curvature

Second-order methods offer superior per-iteration progress but are bottlenecked by the wall-clock cost of Hessian assembly and factorization. In the moderate-dimensional regime where the Hessian fits in memory, the $\mathcal{O}(d^3)$ factorization typically dominates an $\mathcal{O}(nd)$ gradient pass, so a synchronous optimizer stalls at every step: a \emph{synchronization barrier} that negates the benefit of curvature. We propose the \emph{Split-Client} framework, which runs gradient and curvature computation as two concurrent processes. The gradient client never waits; it steps continuously using whatever curvature has most recently been published. The resulting delay $\tau_k$ is a system variable rather than a hyperparameter, and it is time-varying. Our main result is that the wall-clock complexity is governed by the \emph{average} delay $\bar\tau$ rather than the worst-case $\tau_{max}$, via a new Overlap Count lemma. Consequently Split-Client attains $\mathcal{O}(\epsilon^{-3/2}\sqrt{1+\bar\tau})$ wall-clock complexity, matching optimally-tuned Lazy Hessian without knowing $\tau$, and strictly improving when delays fluctuate. Experiments in the moderate-$d$ regime show wall-clock speedups of $25$--$300\times$ over synchronous baselines, growing with dimension.