Parallel-in-Time Variational Inference for Latent Stochastic Differential Equations

Latent stochastic differential equations (SDEs) model continuous-time, irregularly-sampled time series. Training these models typically relies on variational inference (VI) methods that suffer from sequential sampling bottlenecks and compounding integration errors. In this paper, we propose Parallel-in-Time Variational Inference (PiTVI), a framework that parameterizes the variational posterior as a non-Markovian process mapping the driving noise history directly to the local state increments. By leveraging modern sequence modeling architectures such as Transformer and Mamba, the training and inference of the variational posterior become fully parallelizable, reducing the span complexity from $\mathcal{O}(L)$ to $\mathcal{O}(\log L)$. Beyond computational scalability, we demonstrate that this formulation shifts global error propagation from multiplicative compounding to additive scaling. Furthermore, the non-Markovian formulation natively accommodates fractional driving noise, which models physical memory effects and provides a statistical relaxation mechanism when fitting smooth dynamics. We validate these properties across empirical time complexity scaling tests, long-horizon predictions on non-linear systems, and high-dimensional sequence modeling.