Silent Failure Modes in Quantitative Research Agents

Language-model agents increasingly carry out quantitative research in stages spread across sessions. One session proposes an idea and implements it as an analysis, a later session finds and corrects an error in it, another reviews the corrected analysis, and a further session builds on the reviewer's conclusion. The usual safeguard is to recompute the numbers and confirm which files produced them. We study failures that pass these checks because the evidence, although correct, describes a different object from the one the conclusion is about. In our main case, a researcher agent fixed a look-ahead error in an inherited analysis but had not yet reported the fix; a reviewer agent in a new session recomputed the fixed program, found every number correct, and declared the defective original valid without opening it. In 16 frozen states where an inherited defect had been repaired but not reported (one model, tasks we wrote, a custom tool interface), the reviewer judged the original correctly in only 1. A one-paragraph instruction that names the file being judged and asks the reviewer to examine it, or a report format that names each version, raised this to 14 correct judgments, with no wrong judgments and two undelivered reports in each arm. The same kind of failure appears in released software: AlphaAgent's preserved implementation computes a stock's "daily return" as its change from the neighbouring stock, and RD-Agent can reuse an earlier program's evaluation for revised code.