Benchmarking the Detectability of AI-Generated~Proteins

Generative protein models are commonly evaluated using downstream measures, leaving unclear whether their outputs retain detectable traces of their source. We benchmark this property using 1,599 sequences from ProtGPT2, EvoDiff, SwissProt, and nonlearned RandSeq controls. We compare classifiers trained on interpretable sequence features with classifiers trained on ESM-2 embeddings. Interpretable physicochemical and amino acid composition features yield up to 0.92 accuracy, while ESM-2 embeddings exceed 0.96. Both generators resemble SwissProt more closely than the RandSeq controls but remain distinguishable from natural sequences. ProtGPT2 exhibits broader physicochemical variation and greater composition divergence, whereas EvoDiff overlaps more closely with SwissProt. These findings support detectability as an in silico measure of distributional fidelity and reveal generator-specific fingerprints that can guide future experimental evaluation.