MedAdaptAgent: An Automatic Harness for Adapting Medical Imaging Models to New Sites

Reusing pretrained medical-imaging models offers an alternative to training from scratch. A model trained for the same task may perform poorly at a new hospital because imaging protocols and preprocessing differ. Effective adaptation depends on the model architecture, domain shift, and available labels and compute. A model that scores well on a small labeled set may still perform worse on unseen data. We present MedAdaptAgent, an automatic harness that turns a supplied model, target data, and a free-text request into a new-site adaptation workflow. Its LLM-driven Adaptation Planner uses model-site evidence to choose compatible experiments and revise subsequent adjustments. The harness requires evidence of improvement before replacing the original model, then packages the selected model for inference. Evaluation spans 26 segmentation and classification transfers across architectures, annotation budgets, and compute settings. With 20 target labels, mean segmentation Dice rises from 0.483 to 0.774 across 10 transfers, a 60.1% relative gain, with no harmful replacements. Mean classification AUROC rises from 0.810 to 0.837 across 13 transfers, a 3.4% relative gain. Release verification reduces harmful classification replacements by 30.8% versus selecting the highest-scoring model on the available labels, using identical candidates. Public-model cases demonstrate adaptation and reproducible package delivery across contrast and modality shifts. We will release the code on GitHub.