AdapterBench: Does Adapter Shape Matter for Inference-Time Adaptation?

Adapter generation---producing a conditional adapter for a frozen foundation model with a hypernetwork---is an increasingly practical route to inference-time adaptation. Yet recent work has converged almost exclusively on generating LoRA, a representation designed for gradient-based fine-tuning rather than one-shot prediction. Whether the generated adapter's shape matters, and which shape is best, remain untested. We introduce AdapterBench, which isolates that question behind a codec (output structure) and hook-site (attachment point) seam, whose adapters are trained by live end-to-end supervised fine-tuning. Two settings---task-conditioned and document-conditioned adaptation---are evaluated, with six codecs compared in each at their own separately selected operating points. Shape matters, with LoRA appearing as the weakest candidate amongst those tested in both settings. The generated parameter budget of the adapters does not explain these results. These results identify generated adapter shape as an important design choice for agents that must rapidly internalize and reuse new situation-specific information at test time.