Human-Agent Interaction Should Be Evaluated Around Control Opportunities, Not Trajectories Alone
Abstract
This position paper argues that human--agent interaction should be evaluated conditional on control opportunities, not from outcomes or trajectories alone. As agents take on longer and more consequential tasks, the human role shifts toward supervision, verification, and delegation. Outcomes and trajectories show what happened, but they do not necessarily identify when a human could meaningfully alter the interaction. The same non-intervention can therefore reflect calibrated reliance or an unrecognized risk. We operationalize our position through the Critical Control Episode (CCE). A CCE begins when an accessible state change creates multiple feasible responses with plausible downstream consequences. It links this opportunity to human judgment, decision-time evidence, human response, agent or system response, and consequence. The position implies three changes to evaluation: behavioral rates should use eligible opportunities as their denominator; responses should be interpreted with decision-time evidence; and intervention and continued autonomy should receive the same contextual analysis. We specify rules for identifying CCEs, derive implications for evaluation practice, address viable alternative views, and outline a falsifiable validation agenda. CCEs add an opportunity-conditioned episode layer alongside outcomes and agent trajectories.