Benchmarks Are Not Deployments: Geospatial Foundation Models in a City-Scale Dengue Early-Warning System
Abstract
Geospatial foundation models are proposed as a way to supply small-area structure to health systems that lack local data infrastructure. We report two evaluations of one such model inside an operational dengue early-warning system for a large Indian metropolitan authority. Adding embedding features to statistical and machine-learning forecasters left error metrics substantially unchanged across four horizons. On a finer, uniformly sized grid we could not distinguish the embeddings from five open map layers or from a satellite embedding on any of eleven small-area targets: every interval on every pairwise difference contained zero, and the difference fell to 0.001 once the open layers were given the cell's own coordinates. These are failures to detect differences, not evidence of no benefit. A leading contributor is the small number of independent spatial units available at municipal decision scale, and raising a release's resolution does not automatically raise effective sample size at that scale. What did differ was preparation cost, which favours the foundation model and matters most where the ground truth needed to verify it is weakest.