Validity-Constrained Agentic Quantitative Research: A Hidden-Ground-Truth Benchmark and a Bounded Scheduler Comparison

Can an automated quantitative-research process tell planted signal from noise when its final evaluation stays hidden? We introduce qbench, a synthetic benchmark and evaluation protocol for agen- tic cross-sectional factor and model research. A scheduler picks a research direction, an LLM proposes a typed action, and a worker executes it under a fixed 12-attempt budget. The system then com- mits a signed candidate or an abstention before any hidden eval- uation begins. Every submitted claim is then adjudicated against residual returns and measured on a separate sealed test split. True discovery on planted tasks requires both adjudication permission and latent-signal recovery. We compare fixed alternation, uniform- random scheduling, and plain two-armed Thompson sampling with one configured proposer across 40 tasks (24 planted, 16 null) and 144 episodes. No null-task claim survived adjudication, though the marginal one-sided 95% task-risk upper bound remains 17.1% per scheduler. On the predeclared clean-planted stratum, the task-level Thompson-minus-fixed difference was −0.167 (nominal 95% paired bootstrap interval [−0.417, 0.000]). A post hoc pooled comparison gave −0.146 ([−0.292, −0.021]), but the nonzero differences come from only 4 of 24 planted tasks. We did not preregister the paired metric, and the shared proposal cache couples the arms, so this comparison is exploratory. Our contribution is a controlled measure- ment protocol with a bounded case study, not a general scheduler ranking.