From Budget Awareness to Budget Control: Confidence-Calibrated Intervention for Agentic AI

Large language model agents run long trajectories of interleaved reasoning and tool use, and a large part of that computation does not change the final answer. A serving system would like to halt a run once further steps stop buying accuracy. Fixing a token budget in advance spends the same amount whether an answer converged early or is still forming, and stopping when the model's confidence crosses a bar relies on a raw score that is consistently overconfident. This paper studies whether an agent's own answer confidence, once calibrated into a probability of correctness, can drive an online stopping decision, and it asks under which conditions such a decision saves tokens without losing accuracy. By replaying logged agent runs across several model sizes and question answering tasks with tool calls, we reveal three findings. Stopping can help only when a run's budget is dominated by a long tail of low-yield steps, because otherwise every token removed costs accuracy. Calibrating the confidence is necessary before any threshold on its value becomes meaningful, whereas interventions that watch how the confidence changes are insensitive to this miscalibration. Where headroom exists, simple change detectors recover small savings safely, while more aggressive interventions trade accuracy for larger savings. Guided by these findings we propose a lightweight controller that stops only after a minimum budget is spent and after the answer has settled, and that learns which of the eligible long runs to cut. The controller keeps the average accuracy loss near two points across every model and task we test while removing a fraction of tokens, and it avoids the accuracy collapses that undo less guarded interventions.