Distributed Subliminal Learning: Replacing Model Updates with Random-Carrier Outputs

Collaborative learning usually communicates gradients, model deltas, or adapters. Upload therefore scales with trainable parameter count, and heterogeneous knowledge must be combined in weight space. We ask whether related models can instead communicate through behavior on semantically unrelated inputs. We introduce _Distributed Subliminal Learning_ (DSL), a collaborative learning primitive in which each participant adapts a common model, probes it with task-unrelated inputs, and transmits only the resulting _carrier outputs_—logits or sampled completions—which a coordinator pools to distill a shared model. The primitive supports both one-shot model composition and iterative federated learning. The coordinator needs neither local data nor task-related proxy data, and participants never upload model updates. In LLM composition, random-completion distillation achieves $94.56$% preference retention versus $87.76$% for LoRA averaging and yields a $22.0$-point GSM8K gain over the base model versus $0.6$ points for LoRA averaging, with a $30.6$–$49.0\times$ upload reduction. In discriminative federated learning, DSL reaches $96.83$% on MNIST with $8.9\times$ less uplink than FedAvg; CIFAR-10 and Tiny ImageNet expose an accuracy–communication trade-off. These results establish random-carrier outputs as a practical communication primitive for knowledge sharing across distinct collaborative learning paradigms.