Stochastic Lie Bracket Approximations for Zeroth-Order Optimization on Manifolds

We introduce stochastic Lie bracket approximations as a framework for stochastic zeroth-order optimization on manifolds. Classical control-oriented methods exploit deterministic Lie bracket averaging techniques to approximate descent directions along families of spanning vector fields, but their scalability is limited by a restrictive assumption on the orthogonality of dithering signals. We extend this perspective by developing stochastic approximations of Lie brackets. Our analysis shows that the sample paths of these stochastic Lie bracket flows converge, in the mean square sense, to the sample paths of gradient dynamics perturbed by Brownian motion with uniformly bounded coefficients. Due to its geometric nature, the proposed framework is inherently suited for zeroth-order optimization on manifolds, unlike most stochastic approximation techniques which rely heavily on the Euclidean structure. Illustrative examples and numerical simulations are provided to showcase the behavior of the proposed scheme.