You CAN Teach an Old Model New Tricks: Domain Adaptation via Complementary Subspace Expansion

Production retrieval systems require three properties jointly: domain adaptation, preservation of general-retrieval quality, and compatibility with existing vector indices. Existing adaptation methods achieve at most two. Standard parameter-efficient fine-tuning (PEFT) adapts to a domain but breaks the index and degrades general retrieval; backward-compatible training (BCT) preserves the index approximately via distillation while sacrificing general retrieval under domain shift; frozen-backbone side networks (L2C) preserve general retrieval but still modify the deployed embedding via additive combining, so existing indices are not preserved. We introduce \textbf{CoSE} (Complementary Subspace Expansion), a side-branch LoRA whose frozen pathway is structurally untouched, so any index built from the base encoder remains valid --- the adapter is \emph{pluggable}, added on top of an already-deployed index without modifying existing entries. A joint training objective converts the frozen+LoRA concatenation into Pareto-efficient retrieval. Across four distinct domains (Korean broadcast, medical radiology, art paintings, remote sensing) and 3 seeds, CoSE at $6.4$M trainable parameters is the only configuration that simultaneously stays within $\pm 0.048$ of the frozen base on Flickr, COCO, and ImageNet at the deployment-default $\alpha{=}0.5$, is domain-competitive with standard LoRA (ties on RSICD, within $\sim 2\sigma$ on HAN and SemArt, $-0.05$ on ROCO), and is backward-compatible by construction. Deployment-time knobs --- $\alpha$ blending, Matryoshka truncation, partial branching --- expose further cost/quality trade-offs without retraining.