Non-stationary Multi-armed Bandit Algorithms with Time Series Foundation Models

Non-stationary bandit algorithms commonly require hyperparameters that encode the structure or rate of environmental change, and their empirical performance can be highly sensitive to these choices. Setting such hyperparameters typically requires either sufficient historical data or prior knowledge of the environment, both of which are often unavailable in practice. We investigate whether pretrained time-series foundation models can enable robust decision-making across diverse non-stationary bandit environments without environment-specific hyperparameters. A central challenge is that time series foundation models are generally trained with complete time series, whereas bandit policies must select actions to maximize cumulative reward under partial, action-dependent feedback. We use pretrained time-series foundation models to forecast each arm’s next-step reward and act on the forecasts with a simple epsilon-greedy decision rule. Across 948 environments constructed from synthetic processes and real-world datasets, we find that these simple policies can match or outperform a range of non-stationary bandit baselines when environment-specific tuning is not possible.