Timing Replay with a Learned Drift Clock for Language Model Continual Learning

# Timing Replay with a Learned Drift Clock for Language Model Continual Learning Catastrophic forgetting is the main obstacle when a language model must learn a stream of tasks: an update on new data overwrites earlier skills. Experience replay [2] helps, but a uniform buffer spends memory even when the model is stable, while regularizers such as EWC [1] alter the training objective. We instead keep causal next token cross entropy and introduce a *drift clock* that fires replay at expanding intervals in accumulated drift rather than at a uniform rate, using thresholds at $1, 2, 4, \dots$ model days while never resetting $\tau$. A single cue is incomplete: Kullback Leibler (KL) change in output distributions can fire when the model is only becoming more confident, whereas a drop on old task probes (performance drift) can lag until forgetting is already visible. Our contribution is an online learned gate that fuses both cues and is evaluated with one fixed recipe across three continual learning streams. **Approach and setup.** After each task, the same adapter is scored on all tasks seen so far. Metrics are overall performance (OP; mean accuracy after the stream) and backward transfer (BWT; mean change in earlier task accuracy after learning later tasks, where less negative is better). The backbone is Qwen3-0.6B with LoRA [3] on the $q,v$ projections. We use seed 42 and greedy evaluation. Standard CL contains four classification tasks in three orders; Long Seq contains 15 classification tasks in three orders; SuperNI contains 15 instruction tasks in two orders. Memory is a $2\%$ subsample of each past task; KL uses up to 6 snapshot probes and performance drift uses 8 old task probes. The replay baseline advances the clock using adapter weight displacement only. The learned $\lambda$ and $\alpha$ parameters update online on the training stream, with no held out split. KL $\lambda=2$ is the default scale on every benchmark, not selected per stream. **Drift clock.** After each task we store output snapshots and old task accuracy baselines. At a probe step, the cumulative clock is updated as $$ \tau \leftarrow \tau + \ell_2 + \alpha\lambda_{\mathrm{KL}}z_{\mathrm{KL}} + (1-\alpha)\lambda_{\mathrm{perf}}z_{\mathrm{perf}}, $$ where $\ell_2$ is adapter weight displacement from the last snapshot, $\alpha=\sigma(\theta_\alpha)$, and the $\lambda$ values are softplus. The first 24 steps calibrate a model day; a threshold crossing is a dedicated 2 epoch pass over the memory buffer (otherwise train only on the current task). Probe time BCE trains $\alpha$ to predict which cue is larger, while each $\lambda$ scales its semantic cue toward the magnitude of adapter representation drift. Token loss remains causal CE. **Results.** Table 1 reports mean OP / BWT $\pm$ sample standard deviation across task orders (one seed). Replay passes per stream, baseline versus gate, are 21 versus 23, 92 versus 106, and 88 versus 103 for Standard CL, Long Seq, and SuperNI, respectively. KL $\lambda=2$ is one fixed cue on all streams, not a per-benchmark oracle. | Stream | Method | OP | BWT | | :-- | :-- | --: | --: | | Standard CL | Replay baseline | 73.03 ± 1.84 | -5.83 ± 2.39 | | Standard CL | KL lambda = 2 | 73.48 ± 0.88 | -5.15 ± 1.13 | | Standard CL | Combined gate | 73.30 ± 1.29 | -5.17 ± 1.95 | | Long Seq | Replay baseline | 69.89 ± 1.14 | -4.29 ± 1.38 | | Long Seq | KL lambda = 2 | 70.47 ± 1.08 | -4.27 ± 1.26 | | Long Seq | Combined gate | 69.92 ± 1.38 | -4.97 ± 1.76 | | SuperNI | Replay baseline | 43.75 ± 0.80 | -2.21 ± 1.14 | | SuperNI | KL lambda = 2 | 43.33 ± 0.68 | -1.62 ± 1.40 | | SuperNI | Combined gate | 42.86 ± 0.80 | -3.05 ± 0.70 | **Diagnosis.** No single fixed cue-and-$\lambda$ wins all three streams (Standard CL KL $\lambda=4$ is $74.12\pm0.66$ OP; SuperNI Perf $\lambda=4$ is $43.89\pm0.59$); the best single-cue result also uses $\sim24\%$ more replay (26 versus 21 passes), so we do not attribute it to timing alone. The gate replays *more* than baseline on every stream (23/106/103 versus 21/92/88), so the absent benefit is not under replay, though with one seed we can only bound the effect as smaller than the across order spread. It does not collapse to a hard switch (end of stream $\alpha\approx0.46$ on Standard CL and $\approx0.32$ on longer streams, leaning toward the performance cue), yet this correct leaning produces no OP gain and, on SuperNI, a mild regression despite $\sim17\%$ more replay (103 versus 88 passes). This isolates the probe-time objective, rather than the two channel premise, as the failure point: $\alpha$ is supervised to predict cue-magnitude order, a proxy only loosely coupled to replay timing quality. ## References 1. J. Kirkpatrick et al., “Overcoming catastrophic forgetting in neural networks,” *PNAS*, 2017. 2. D. Rolnick et al., “Experience replay for continual learning,” *NeurIPS*, 2019. 3. E. J. Hu et al., “LoRA: Low-rank adaptation of large language models,” *ICLR*, 2022.