AMP Challenge: A Benchmark for Generative AI in Antimicrobial Peptide Design

Antimicrobial peptides (AMPs) are a promising class of therapeutics against antibiotic-resistant infections, and generative AI has recently emerged as a powerful tool for their design. Yet most proposed methods are not validated in the wet lab. Experimental testing is costly, and the few studies that do report it use incompatible training data, bacterial strains, assay conditions, curation pipelines, and success criteria. Cross-study comparison is therefore impossible, and the field has no shared signal on which generative approaches actually work. This raises the central question that AMP Challenge is built to answer: which generative modeling strategies, computational metrics, and candidate-selection rules predict experimental antimicrobial efficacy? AMP Challenge is the first standardized benchmarking competition for generative AI applied to AMP design, in which computational predictions are directly falsified by wet-lab outcomes, and where progress claims in generative AI can be evaluated against reality rather than against other models. In the (i) computational phase, all submissions are scored under a fixed pipeline, and up to 20 qualifying teams advance. In the (ii) experimental phase, 25 peptides per team (up to 500 peptides total) are synthesized under a single vendor workflow. Minimum inhibitory concentration (MIC) and half-maximal hemolytic concentration (HC50) assays are performed in a single laboratory across 20 clinically relevant Gram-negative and Gram-positive strains, including 8 multi-drug resistant (MDR) isolates. Teams compete across five categories: broad-spectrum, Gram-positive, Gram-negative, MDR, and selectivity. Deliverables include an open benchmark dataset pairing computational metrics with standardized experimental MIC/HC50 data, a public model repository, and a jointly authored benchmarking paper. The benchmark will remain open after the competition closes, enabling continued evaluation of new methods against the same experimental ground truth. By releasing both successful and unsuccessful peptides, the competition directly counters the positive publication bias that has shaped the field to date.