Cross-Family Portability of Interpretable Features in Protein Language Models

Sparse autoencoders (SAEs) provide an enticing way to understand the internal mechanisms of protein language models (PLMs), using interpretable concepts for functional discovery and sequence generation steering. Using residue-level Swiss-Prot annotations, this work demonstrates that SAE features selected in one protein family transfer poorly to other families (median imported average precision $0.019$), while the annotations remain independently decodable. Through decomposition, we find that each family--concept representation has a reproducible residual that predicts feature portability between families. Reducing this residual in model activations before SAE training increases feature portability, while amplifying it decreases portability. We further demonstrate steering across $17$ protein families, increasing asparagine allocation to glycosylation-sequon-compatible contexts by $7.6$ percentage points beyond a uniform amino-acid bias. These findings advance mechanistic interpretability and steering of PLMs by showing how family-specific representations limit feature portability and why biological interpretations must be validated across protein families.