Attribution Without Action: Why Gradient-Based Data Curation Fails for Cross-Protein Fitness Prediction

Data curation has improved model training in natural language processing and vision, and gradient-based data attribution offers a principled way to identify training examples that help or hurt a target prediction. We test whether this premise transfers to cross-protein variant-effect prediction from deep mutational scanning (DMS) data. We fine-tune ESM-2 to predict fitness effects for held-out proteins, reaching a cross-protein macro Spearman correlation of 0.470. Because computing TRAK attribution for the full-fine-tuned model is prohibitively expensive, we use a LoRA-fine-tuned ESM-2 model as an attribution surrogate and use its scores to construct global and target-specific curation recipes. The recipes fail: the global recipe yields a negligible gain (+0.004), while all matched target-specific recipes reduce macro Spearman by approximately 0.014-0.015. Attribution is moderately reproducible across independently fine-tuned ESM-2 and ESM-1v models ($\rho=0.433$) and yields individual biologically interpretable relationships, but shows little association with sequence identity, embedding similarity, assay category, Pfam or Gene Ontology overlap, or structural similarity. A direct leave-one-training-protein-out retraining check gives a positive but underpowered and statistically non-significant association between attribution and removal effects ($\rho=0.567$, $p=0.112$, $n=9$). These results expose a gap between descriptive attribution and effective intervention: reproducible attribution structure does not necessarily provide an actionable basis for data curation. We hypothesize that protein-level filtering removes too much useful supervision relative to its benefit and that curated DMS data contain fewer readily identifiable harmful examples than the heterogeneous datasets in which attribution-guided filtering has previously succeeded.