What Does the Student Get to Ask? Adaptive State Exposure in Knowledge Distillation

Knowledge distillation can transfer only teacher behavior expressed on states used for supervision. We ask whether the source of those states determines which teacher capabilities transfer. In a controlled sequential task, two teachers agree on common inputs but differ in one latent capability. For the primary comparison, we hold the models, paired initializations, optimization, and total supervision fixed, and vary only whether queries are frozen from the initial student or refreshed as the student learns. Across 50 paired seeds, adaptive queries raise held-out accuracy from 2.4% to 27.6%; querying a capability-ablated teacher yields 0%, tying the gain to the teacher’s distinctive behavior. Queries frozen from a mature student match the adaptive result, so continuous query refresh is unnecessary. At a fixed source checkpoint, greater exploration increases teacher-distinguishing exposure and transfer. Transfer occurs on only three of four structures. In a pre-specified follow-up on the failed structure, students learn the missing local action but still fail the full action sequence. Thus, query construction is part of the distillation intervention: source maturity and state coverage govern what can transfer, while local action coverage need not support composition.