A Finite-Rate Selection Law for Annealed Gibbs Sampling

A sampler can generate plausible examples in the wrong proportions. We show how this happens in annealed Gibbs sampling, which updates one variable at a time while lowering the temperature. In the Curie--Weiss model, binary variables tend to share one sign, giving two modes, and a weak field makes one mode more probable. Each run commits to a mode near the critical temperature, while the field still competes with random fluctuations, and rarely switches afterwards. The samples therefore keep only a fraction of the model's log-odds between the modes. We turn the classical probability of this commitment, due to Kondepudi and Nelson, into a law for the fraction that agrees closely with an exact calculation of the sampler. The law also explains why slower cooling helps little: on a linear ramp, cooling sixteen times more slowly only doubles the fraction. Reheating runs that settled away from the favored mode gives them a second choice, and the exact calculation shows that it again keeps only a fraction. In a Hopfield model, competition among stored patterns raises this fraction; a Gaussian approximation of the competition predicts the result to within 11% across pattern counts and sizes.