PCAF: Bounded Exact-Successor Memory for Efficient Long-Context Models

Efficient causal models compress long histories, but a compact state may forget which token followed an earlier occurrence of the current one. We study PCAF-Exact, an exact-key formulation of the Parallel Causal Associative Field (PCAF) that retains up to eight observed successors per token and mixes their sparse votes with a parametric language model. Records are displaced by recurrences of the same key rather than distance alone. Across three-seed PG-19 studies, the learned-readout model lowers perplexity from 129.72 to 114.54 against capacity-matched reader-free controls at 40M/64K, with gains also at 125M/32K and 350M/8K. Removing the reader or permuting successor values reduces its benefit, while 8K controlled retrieval and overwrite accuracy exceed 99.9%. A co-trained fixed-rank readout retains reader gains without learned scoring. In a separate pretrained study, a task-aligned successor selector improves 32K multikey accuracy from 73.96 to 95.57% with frozen Qwen3-1.7B adapters. Broader WikiText cache reads improve perplexity modestly but require 5.5–7.6 times measured reader time. These results identify a quality–cost niche for bounded exact successor memory.