Better Thinkers, Worse Believers: Does Self-Distillation Reduce Chain-of-Thought Faithfulness?

Self-distillation methods that let a single language model act as both teacher and student are an appealing alternative to reinforcement learning and external-teacher distillation for improving reasoning. We ask whether one such method, On-Policy Self-Distillation (OPSD), affects the faithfulness of chain-of-thought (CoT) reasoning relative to a standard supervised fine-tuning (SFT) baseline. Using Qwen3-1.7B on 998 paired GSM8K problems, we evaluate a base, an SFT, and an OPSD model under six diagnostic conditions covering clean accuracy, misleading and authority-based hints, irrelevant context, conversational sycophancy, and biased few-shot demonstrations. OPSD attains the highest clean-math accuracy (92.0\%), but its accuracy collapses to 46.0\% when a plausible wrong hint is added, compared to 70.0\% (base) and 73.0\% (SFT); its rate of flipping a correct answer to a wrong one under misleading context (SC2W) is 46.0\%, roughly $2$--$3\times$ higher than the base and SFT models. An aggregate six-axis faithfulness score ranks OPSD lowest (80.2\%) and SFT highest (89.8\%). We further show that all three models internally notice the contradiction introduced by a wrong hint (100\% reasoning-conflict-loop rate), but OPSD is more likely to resolve that conflict in favor of the hint rather than its own calculation. Our results suggest that capability gains from on-policy self-distillation can come at the cost of robustness to untrustworthy context, and that CoT faithfulness should be evaluated alongside accuracy when using self-distillation for post-training.