What Post-Training Suppresses and What Persists: A Controlled Study of Anti-Muslim Bias Across LLM Development Stages
Abstract
That language models exhibit anti-Muslim bias at the output level is well established, yet the point in the training pipeline where this bias enters, and the stage that actually suppresses it, remains unclear. We address this gap with a small-scale controlled pipeline in which corpus composition is treated as an experimental factor. Starting from a single base model (Qwen2.5-0.5B), we fine-tune three branches on corpora matched for size (2,361 documents each) but differing only in the proportion of Muslim related text carrying a negative association (Bdata = 0.41, 0.50, 0.75). Each branch then receives identical instruction tuning and identical supervised safety training, and all ten resulting models are evaluated on one fixed 300-prompt benchmark. Varying the corpus produces no detectable effect: none of the between-branch differences survive Holm correction at any stage, and the widest gap observed is only 3.3 points. Instead, the change comes entirely from post-training. Simply exposing the model to Muslim related text, regardless of its composition, pushes the bias rate up from 6.7% to between 8.3-11.7%; instruction tuning subsequently drives it down by 60–79% (a 9.2-point drop in the balanced branch, pHolm = 0.002). What remains afterward is category-specific violence and terrorism associations are fully eliminated in every branch, whereas broader cultural and values based generalisations about Muslim communities are the only bias category that stays non-zero even after safety training