Does Benchmark Correlation Predict Selection Quality in Protein Design?

Protein-ranking methods commonly report rank correlation with experimental measurements, while protein-engineering campaigns act on a small set of top-ranked candidates. We test how well global rank correlation predicts the value of this selected shortlist across four corpora with experimentally measured protein function. On 10 published plastic-degradation studies, selecting the top 20\% of variants with a protein language model captures essentially none of the achievable gain: selection utility is $-0.016$ for ESM-2 and $-0.006$ for ESM-C. Other corpora show substantially higher utility, indicating that protein-language-model rankings can support useful selection in some experimental regimes. Rubisco provides the strongest positive case. ESM-2 reaches Spearman $\rho=+0.631$ against maximum turnover across 8{,}760 single mutants, but restricting the analysis to well-measured variants lowers the correlation to $+0.364$, and controlling for wild-type residue preference lowers it to $+0.259$. The relationship between correlation and selection utility also depends on benchmark family. At the shared top-20\% cutoff and matched correlation, SSMuLA combinatorial libraries provide $+0.115$ more utility than ProteinGym deep mutational scans under ESM-2, and ESM-C reproduces the offset. These results show that global correlation alone does not determine the value of the candidates selected for experimental testing. Evaluation should measure selection quality at the cutoff relevant to the intended campaign and validate that measurement on data generated under a comparable experimental regime.