Large language models are increasingly built the way we build software — as modular systems rather than monolithic checkpoints. This talk presents Granite Switch, IBM Research's toolchain for composing independently developed capabilities into one deployable Granite model that switches between them on demand, at production inference speed. A single checkpoint runs on both vLLM and HuggingFace, with no retraining to combine functions. Switching rests on three ideas: control tokens that activate a function from their position forward, a normalized KV cache so functions don't interfere, and per-token-position weight selection that reuses prefill instead of recomputing it.

The core of the talk is a tour of the types of module support Granite Switch composes into that one checkpoint, all sharing the same index space and switch: LoRA adapters as switchable functions; Shadow Residual, a substitution-site adapter acting at a specific marker position; and activation classifiers, classification heads that ride the same switch and emit their verdict as an ordinary generated token with no separate classifier model and no extra inference call, enabling in-model fast decisions such as safety, hallucination detection, and answerability.

Lastly, we will provide another angle towards this modularity by connecting to vLLM-Hook, our open-source plug-in that exposes model internals such as attentions and activations for probing and intervention without retraining, offering internal signal reading and monitoring on a running model.