On-the-fly Weight Generation: A Hypernetwork Proof of Concept on ARC-1D

General-purpose models can adapt to many tasks from context, while specialised models can execute individual functions with substantially less capacity. Yet obtaining such specialists requires task-specific training or adaptation. We ask whether they can instead be generated directly from a few demonstrations. Using ARC-1D as a controlled testbed, we show that individual transformations admit tiny specialist models, and that a hypernetwork can generate such models directly from demonstrations. The generated parameters form a structured weight space, while the resulting specialists generalise across instances and show partial compositional generalisation to unseen combinations of known transformations, particularly without explicit task identifiers. Together, these results provide a proof of concept that few-shot task context can be compiled on-the-fly into compact executable model parameters, and that the resulting weight space can support reuse and partial compositional generalisation.