Online Active Testing for Budget-Efficient Evaluation of Molecular Potentials

Reliable evaluation of machine learning models in quantum chemistry can require a large number of expensive reference calculations. This challenge is particularly acute when molecular configurations are generated sequentially: each new configuration requires an immediate decision on whether to perform the quantum-mechanical evaluation, which can be orders of magnitude more expensive than generating the configuration itself. While active learning methods address this computational bottleneck during model training, existing active testing methods largely assume access to a pool of unlabelled samples and are therefore not directly suited to sequentially generated configurations. We introduce \emph{Online Active Testing} (OAT) for estimating model risk when data arrive sequentially, and reference calculations are available only under a limited budget. OAT combines an unbiased importance-weighted estimator with a lightweight surrogate that guides evaluation decisions online. We derive the variance-optimal sampling rule and evaluate OAT on a molecular simulation task involving expensive quantum-chemistry calculations. On this task, OAT achieves the same estimation accuracy as uniform sampling using only $\sim$30\% of the reference calculations, compared with $\sim$23\% for an oracle strategy with access to the true loss. These results demonstrate that OAT can substantially reduce the computational cost of reliable model evaluation.