Attempted-Only Accuracy Favors Low Coverage
Abstract
Many reputation schemes report correct answers divided by attempts. Refusals never enter this ratio, so an agent can protect its score by avoiding tasks it may miss. We ask what happens once such scores determine which strategies are copied. In a two-level task model, attempted-only accuracy is convex in hard-task coverage and has no interior optimum. With a bonus (w) for a correct hard answer, the preferred endpoint changes at (w^*=(1-p)/p), where (p) is hard-task success probability; the easy-task fraction cancels. Finite-population simulations recover the two regimes over 20 seeds and eight ((q,p)) settings. With heterogeneous ability, reputation and ability have correlation (0.225\pm0.007) under attempted-only accuracy and (0.887\pm0.006) when every assigned task remains in the denominator. We repeat the selection experiment on cached answers from four language models over 300 GSM8K problems. Coverage is (0.062\pm0.002) under attempted-only accuracy and (0.866\pm0.005) under the all-assigned score; the corresponding reputation–ability correlations are (-0.105\pm0.055) and (0.901\pm0.016). Coverage is imposed in this experiment, so it isolates the denominator effect rather than learned abstention. The result exposes a simple failure mode: with solved items held fixed, reputation can rise merely because an agent’s assigned set shrinks.