Mixture-of-Experts for Online Matrix Completion on a Drifting Union of Subspaces

Many partially observed data streams are globally high-rank yet locally low-rank: user–item interactions in recommender systems span multiple preference groups, distinct operating regimes in sensor and network monitoring induce different low-dimensional structures, and the active latent subspace in adaptive systems shifts over time. We formalize this as online matrix completion on a drifting union of subspaces: each sample lies in one of K low-dimensional subspaces, only a random subset of its coordinates is observed, and the subspaces drift across epochs. We propose MoSAIC (MoE Subspace Adaptation via Incremental Completion), a method based on a routed mixture of low-rank experts trained in two phases. A base model is first pre-trained on samples from a fixed source distribution. It is then adapted on the non-stationary stream by routing each incoming sample to a single expert and updating only that expert. Our analysis identifies sufficient conditions under which the router remains correct with high probability throughout learning, while within each epoch the routed experts contract toward the current subspaces at a sublinear rate. The technical core is a uniform routing concentration argument that converts the random time steps on which each expert is updated into a deterministic time scale, reducing the per-expert analysis to a tractable stochastic recursion. Experiments on streams with repeated non-stationary changes corroborate the theory and show clear improvements over competitive baselines.