Known by Construction: Calibrating Model-State Instruments Against Models Whose State Was Installed, Hidden, or Never Present
Manan Wadhwa ⋅ Shivam Dubey
Abstract
Instruments that claim to read a language model's internal state (self-report, forced choice, activation directions, hypothetical trade-offs) are validated by their agreement with one another. No case exists where the answer is known independently of them, so a shared confound would pass every check. We build that case from LoRA fine-tunes of Qwen3-4B in a grid world with a silently penalised tile: an avoidance habit installed by reinforcement (on four of eight seeds), installed and then hidden by an anchored fine-tune that teaches a one-sentence remark, or never installed (the remark alone), plus a sham (the same reinforcement on a permuted reward, installing nothing) and a decoy (reward swapped, so the policy avoids the other tile). Hidden models match the untouched model's greedy behaviour on all eight seeds. Placebo instruments were chosen before any model existed and verdict rules committed before the cells they judge were built. Every instrument reads the prompt or the training history, not the state. An in-context instruction moves the verbal instruments by up to 9.5 logits; a hidden habit moves them by at most 2.2, mostly attributable to the fine-tune. The activation direction and the trade-off instrument separate the sham from remark-only models as far as they separate the hidden habit: they detect that an optimiser ran. Relearning recovers avoidance from the decoy as readily as from the hidden habit. The one in- domain direction that passes the pre-committed rule reads at the hidden models' level ($+0.97$ against $+1.03$) on models that inherit the habit's adapter but have had the habit extinguished, consistent with adapter lineage rather than the habit, though a residue is not excluded. Agreement among instruments that share a cause is not validity; we release rows, scorers and recipes so an instrument can be calibrated where the answer is known before it is trusted where it is not.
Chat is not available.
Successful Page Load