RecAudit: Completion-Safe Agentic Diagnostics for Video Recommendation Systems
Sameer T Khanna ⋅ Sohini Roychowdhury
Abstract
Agentic-AI runtimes budget for tokens, latency, and concurrency but treat the number of decisions requested per call as implicit and unmonitored. Across 20 calls requesting 128 or 256 verdicts from a video-recommendation diagnostic pipeline, none returned a verdict, despite being well-formed, non-refusing, non-truncated responses within the output cap resulting in \emph{silent cardinality collapse} (SCC). A crossed intervention isolates the cause: at fixed prompt length, completion collapses as requested decisions grow, while at fixed request count, it remains stable as prompts grow. A second stack reproduces the zero-completion result. RecAudit keeps model output provisional behind two code-owned commits: verdicts require same-case citations and contract-required observations, and jobs require a terminal record for every requested key. On 256-case jobs both modes return every key (yield varies $\leq$2.3\,pp). On a 475-case expert-decision replay, RecAudit gains 11.2-19.6 accuracy points at 80.0-84.2\% coverage with zero invalid citations and zero observation violations. The design makes cardinality an explicit orchestration input and completion a runtime post-condition.
Chat is not available.
Successful Page Load