The Capacity of In-Context Binding: a Scaling Law, a Recipe Split, and its Boundary
Manas V Sai Ravulapalli ⋅ Samrath S Chadha
Abstract
How many entity–attribute bindings can a language model hold in context before recall falls? We measure capacity two ways: continuous recall-versus-load curves for 12 models at or below 3B, and a threshold sweep over 30 open models to 12B parameters. The continuous curves follow K50 = cNα with α= 0.820 and R2 = 0.73, and the sweep spans an 8×range split by pretraining recipe. On the continuous curves, the recipe coefficient spans zero after controlling for scale. Direct task training reaches 26×the zero-shot prediction, and its formation cost follows a power law of the same form in two independent codebases.
Chat is not available.
Successful Page Load