Efficient Modeling of Irregular Time-Series with Stochastic Optimal Control

Many real-world datasets, such as healthcare, climate, and economic data, are often collected as irregular time series, which pose significant challenges for modeling. Previous research has approached this problem in two main directions: 1) Transformer-based models and 2) dynamics-based models. Transformer-based models efficiently handle irregular time series with simple architectures and time encoding but struggle with long sequences and require many parameters due to the lack of inductive biases. Continuous dynamics-based models offer accurate Bayesian inference of dynamic states but suffer from the complexity of sequential computation, leading to increased computational costs scaling with the length of time intervals. To address these limitations, we propose Parallel Bayesian Diffusion Filtering (PBDF), a variational inference algorithm based on parallelizable stochastic differential equations and stochastic optimal control theory. PBDF combines the parallel inference capabilities of Transformer-based models with the Bayesian inference of continuous-discrete state space models. Through empirical evaluations on the USHCN and Physionet datasets for both interpolation and extrapolation tasks, we demonstrate PBDF’s superior performance and computational efficiency.