A Hamiltonian Dynamics Perspective on Accelerated Methods in Optimisation

In this work, we establish novel connections between Hamiltonian dynamics and accelerated methods in optimisation. We do so by showing particular numerical integrators for a specific Hamiltonian dynamics correspond to limiting versions of classical accelerated methods, namely Nesterov’s AGM and Beck and Teboulle’s FISTA) complementing a known connection with the heavy ball method of Polyak. Then, we show that when these integrators are embedded within an integrate-process-reset scheme with suitable choices of integration times and processing methods, the resulting algorithms achieve the same accelerated iteration complexity matching their classical counterparts for minimising convex functions. These results suggest a new perspective on designing fast methods for optimisation, wherein momentum schedules in classical accelerated methods can instead be represented through the duration and processing of Hamiltonian trajectories.