Understanding the Bitter Lesson in Time Series Foundation Models

In this talk, we discuss the bitter lesson in designing time series foundation models (TSFMs). First, we introduce Chronos and Chronos-Bolt models and how they differ in their design choices. Importantly, we use these models to more broadly represent general design choice differences in TSFMs, e.g., patch size, continuous vs. quantization embedding, and regression vs. classification loss function. We then show that while Chronos-Bolt, which has more natural time series inductive biases, e.g., continuous embedding and quantile loss function, performs better on classical time series benchmarks, Chronos performs better on chaotic systems. We then identify biases induced by these design choices, e.g., temporal, geometry and regression-to-the-mean biases to explain what is causing these different behaviors and the pros/cons of each design choice. Lastly, we conclude with forward looking view on TSFMs and the newly-released multivariate Chronos-2 TSFM.