Which Eviction Policy Should an LLM Cache Use? A Systematic Study Across Workloads, Capacities, and Encoders

Semantic caches answer an incoming query with the response stored for a nearby embedding, but proposed eviction policies have not been compared under one protocol. We evaluate FIFO, LRU, LFU, ARC, GDSF, a streaming adaptation of SISO, and a semantic-redundancy policy across three ordered query corpora, three cache capacities, and two encoders. No evaluated policy improves on LFU by more than 0.041 percentage points in any of the eighteen settings. Replacement still matters at tight capacity: FIFO and streaming SISO trail LFU by as much as 8.7 points. We explain the missing upside with a conditional packing result. Under exact lookup and insert-on-miss, a newly inserted entry cannot have a resident neighbor within the hit radius, so a geometry-aware eviction rule receives little new redundancy signal. A separate audit finds that hit validity is a larger problem than policy choice. At a median-nearest-neighbor operating point, only 2.1–3.9% of sampled LMSYS and QQP hits are judged answer-substitutable, reducing raw hit rates of 51–60% to quality-adjusted rates of 1.1–2.2%. LFU is the strongest simple default in this protocol, but threshold calibration and hit validity require attention before sub-point eviction differences.