Timezone: »

 
Oral
SLIP: Learning to Predict in Unknown Dynamical Systems with Long-Term Memory
Paria Rashidinejad · Jiantao Jiao · Stuart Russell

Tue Dec 08 06:15 AM -- 06:30 AM (PST) @ Orals & Spotlights: Dynamical Sys/Density/Sparsity
in Orals & Spotlights Track 06: Dynamical Sys/Density/Sparsity »

We present an efficient and practical (polynomial time) algorithm for online prediction in unknown and partially observed linear dynamical systems (LDS) under stochastic noise. When the system parameters are known, the optimal linear predictor is the Kalman filter. However, in unknown systems, the performance of existing predictive models is poor in important classes of LDS that are only marginally stable and exhibit long-term forecast memory. We tackle this problem by bounding the generalized Kolmogorov width of the Kalman filter coefficient set. This motivates the design of an algorithm, which we call spectral LDS improper predictor (SLIP), based on conducting a tight convex relaxation of the Kalman predictive model via spectral methods. We provide a finite-sample analysis, showing that our algorithm competes with the Kalman filter in hindsight with only logarithmic regret. Our regret analysis relies on Mendelson’s small-ball method, providing sharp error bounds without concentration, boundedness, or exponential forgetting assumptions. Empirical evaluations demonstrate that SLIP outperforms state-of-the-art methods in LDS prediction. Our theoretical and experimental results shed light on the conditions required for efficient probably approximately correct (PAC) learning of the Kalman filter from partially observed data.

Keywords: Bandit Algorithms · Learning Theory · Algorithms · Algorithms -> Online Learning; Theory

Author Information

Paria Rashidinejad (University of California, Berkeley)
Jiantao Jiao (University of California, Berkeley)
Stuart Russell (UC Berkeley)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors