Quantization and Multilingual Safety: Does 4-bit NF4 Disproportionately Degrade Refusal Behavior in Low-Resource Languages?
Abstract
Post-training quantization enables efficient deployment of large language models, but its effects on multilingual safety—particularly in low-resource languages—remain unclear. We conduct a pre-registered audit of 4-bit NormalFloat (NF4) quantization in Qwen3-4B and Llama-3.1-8B-Instruct across 10 languages, evaluating multilingual capability and safety refusal as distinct constructs. NF4 substantially degrades Llama-3.1-8B capability, with losses increasing from −9.4 pp in English to −13.7 pp in low-resource languages; a Page trend test confirms the monotonic gradient is statistically significant (L=52,307, p<0.001, supporting H2). For the pre-registered safety-refusal interaction (H1), the critical Llama NF4 low-resource stratum suffers widespread output degeneration (91.9% degenerate; 0/945 outputs contain actionable harmful assistance under blinded adjudication), rendering automated refusal scores unreliable for that stratum. Across all non-degenerate strata (46,487 observations), a pre-registered GLMM finds no evidence of a low-resource-specific safety interaction (β=0.027, p=0.88), a result we explicitly frame as conditional on output quality. Qwen’s high baseline LRL loop rates (40.2% BF16, 50.5% NF4) similarly confound refusal scoring, corroborating that apparent low-resource safety collapses reflect generation breakdown rather than jailbreak vulnerabilities. Blinded validation (n=200) and tri-stream sensitivity analysis confirm these findings.