Autoregressive Protein Sequence-Structure Co-Design with Modeling-Free Tokenization

Protein sequence-structure co-design is dominated by continuous diffusion/flow models and discrete protein language models (PLMs) using VQ-VAE structure tokens. However, diffusion models lack tractable per-step likelihoods, limiting test-time search, while VQ-VAE-based PLMs inherit codebook collapse and lossy geometry reconstruction. We instead cast co-design as autoregressively generating residues in 3D. At each step, a decoder-only Transformer predicts an amino-acid identity together with a discretized backbone frame. We introduce a modeling-free tokenizer that maps each frame to four geometric tokens, achieving faithful reconstruction without a learned structure decoder. We further train autoregressive models and diagnose two failure modes of vanilla AR co-design: secondary-structure entropy asymmetry that causes pathological helix extension, and causal myopia that lacks global planning, both can be mitigated by simple tricks of length-prefix conditioning and transition-aware loss reweighting. With likelihood-based decoding and partial rollout that are unique to the autoregressive formulation, our model reaches strong in-silico results that are competitive with existing PLM baselines.