Same Models, Fewer Accelerators: Multi-Tenant LLM Post-Training via Co-operative Time-Slicing

Accelerator scarcity now shapes how machine-learning teams work: post-training jobs — RL fine-tuning, supervised fine-tuning — queue for a fixed pool of accelerators, even as the accelerators they wait for sit idle 30-74% of the time in measured production RL runs [1]. The idleness is structural. Trainers and samplers typically run on disaggregated pools because their workload characteristics differ completely — training is compute-bound, autoregressive generation is memory-bound — and disaggregation lets each scale independently. But in on-policy RL, training and generation depend on the same policy version, so the phases strictly alternate: while one pool works, the other waits. Off-policy variants let the phases overlap, but generation is typically the slower, costlier phase, so trainers still stall into recurring idle windows. Accelerator sharing is the natural remedy, but until now it has been practical only for adapter-based tuning: LoRA states are small, and mainstream training and serving stacks expose ready APIs to offload and load adapters, so multiple tenants can swap cheaply. Full-parameter fine-tuning — where each tenant's weights, optimizer state, and caches fill entire accelerators — has had no equivalent. This raises the question: can full-parameter tenants share accelerators without changing what the models learn? Co-operative time-slicing turns RL phases into schedulable units. Each tenant's full accelerator state — weights, optimizer state, KV cache — parks in host memory while the tenant idles and is restored when its next phase begins; jobs signal those transitions through two minimal RPCs — acquire and yield — and the runtime grants tenants exclusive accelerator access in turn. The same mechanism serves synchronous RL, where trainers and samplers interleave, and asynchronous RL, where only trainers time-slice — reclaiming the fragmented idle gaps while samplers run uninterrupted. Phase-level multiplexing has precedent: a proprietary production stack demonstrated it for synchronous, on-policy jobs within its own training framework [2], and our own initial benchmarks in the synchronous regime raised aggregate accelerator duty cycles from a ~40% baseline up to 70% without impacting convergence or accuracy. We generalize the mechanism to unmodified, industry-standard RL frameworks, and show how to make it multi-tenant for a fine-tuning-as-a-service platform — tenants with different frameworks, base models, and objectives, including non-RL supervised fine-tuning. We evaluate multi-tenant consolidation with three heterogeneous tenants — text-to-SQL RL (Spider), math RL (GSM8K), and dialogue supervised fine-tuning (MultiWOZ), each fine-tuning a different base model — sharing two GPUs (one training pool, one sampling pool) versus six dedicated GPUs. The shared GPUs sustain more than 2x the duty cycle of the dedicated baseline — provisioning two-thirds fewer GPUs and paying for ~38% less GPU-time than the same jobs running concurrently on dedicated hardware. The overhead of slicing GPUs among tenants is small: median context switches of 0.5-1.6 s and median queue waits of 1-15 s. The kicker is that convergence is unharmed: the text-to-SQL tenant climbs from 25% to 43% response accuracy, matching its dedicated-GPU baseline; the math tenant roughly doubles its eval accuracy, tracking its baseline; and the SFT tenant's eval loss falls from 4.77 to 0.56, point-for-point identical to its baseline. The collective workload finishes only 1.9x slower than concurrent runs on six dedicated GPUs. Phase-aware co-operative time-slicing makes full-parameter fine-tuning multi-tenant, converting idle time into throughput without changing what models learn — reshaping the economics of RL post-training infrastructure. [1] K. Chen, K. W. Cheang, and D. Patel. RL Systems Mind the Gap: Matching Trainer and Generator Throughput. SemiAnalysis, June 2026. [2] T. Wu et al. RollMux: Phase-Level Multiplexing for Disaggregated RL Post-Training. arXiv:2512.11306, 2025.