Generative modeling of intrinsically disordered protein regions by reinforcing sparse autoencoder features

Intrinsically disordered protein regions (IDRs) are ubiquitous across all kingdoms of life. These structurally heterogeneous regions play central roles in cellular processes such as transcriptional regulation, cellular signaling, and subcellular localization, yet they have remained largely inaccessible to generative design methods. This gap persists because structure-based generative approaches are inapplicable to proteins that lack a stable fold, and because existing protein language models are trained on full-length protein sequences, thus learning a prior which is biased towards folded domains. Here, we present IDiom, an autoregressive protein language model trained on 54 million IDRs curated from the AlphaFold Database that enables the generation and design of diverse and biologically realistic IDR sequences. In addition, we introduce reinforcement learning with sparse autoencoder (SAE) features (RL-SAE), in which the model is rewarded for firing a specified set of SAE features. Together with IDiom, this enables the interpretable and composable design of IDRs which carry the essential sequence motifs and predicted functionality of natural IDRs.