When Wording Changes the Control Set: Frame-Conditioned Evaluation of Instruction Interference
Nok Man Chan
Abstract
Prompt sensitivity is usually treated as an outcome-robustness problem: change the wording of a task, and a language model's measured performance can change. We study a different failure that occurs one stage earlier. In matched-reference evaluations, wording can change which controls count as matched and therefore which comparison is well defined. We study pair-specific instruction interference, where the question is whether adding instruction $A$ reduces compliance with instruction $B$ more than comparable companions do. The usual comparison between $B$ alone and $B{+}A$ changes both instruction count and companion identity, so we instead compare $B{+}A$ with $B{+}R_j$ for same-form references matched to $A$ on chance-corrected stand-alone difficulty. That matching cannot be pooled across wording. On one model, stand-alone compliance with the same target requirement at fixed severity is $0.22$, $0.68$, $0.42$ and $0.52$ under four sentence frames, and within-model frame variation in the matching coordinate can exceed the registered $0.10$ caliper. We therefore condition the operating points, difficulty coordinate, admissibility test, reference set and matched-reference contrast $\Delta_R(f)$ on wording frame $f$. At equal measurement budget, frame-conditioned reference identification changes the admissibility verdict on two of fourteen supported model $\times$ frame combinations and changes one model's eligibility. Frame-level verdicts agree across re-executions on all fourteen frames, while candidate sets and selected reference triples turn over substantially, and the operating-point rule can move at band edges under decoding noise alone. The contribution is methodological: linguistic realization can affect the validity of an evaluation's control set before the focal composed outcome is observed. No confirmatory composed observation exists under this design, so we report no value, sign or interval for $\Delta_R(f)$.
Chat is not available.
Successful Page Load