Alignment Is Not Separation: Two Geometric Axes of Cross-Lingual Sentiment Representation in a Multilingual LLM
Sai Adith Prakash
Abstract
Activation steering and refusal-direction interventions are increasingly proposed as lightweight, training-free tools for improving multilingual LLM behavior, yet their reliability varies sharply and unpredictably across languages. Prior work on cross-lingual refusal directions reports that safety-relevant interventions transfer well for some non-English, low-resource languages but fail unexplainably for others (e.g., Arabic), with the failure attributed only loosely to ``a geometric mismatch.'' We investigate this puzzle directly by extracting difference-of-means sentiment directions for six languages spanning two resourcedness/ tokenization-fragmentation categories (English; low-resource Yoruba, Igbo, Amharic; high-resource-but-heavily-fragmented Arabic, Hindi) from Qwen2.5-7B-Instruct, and measuring two distinct geometric properties: (1) directional alignment with the English sentiment direction (cosine similarity), and (2) within-language cluster separation quality (Silhouette score) between positive and negative sentiment activations. We find that these two properties are only moderately correlated ($r=0.79$, $n=5$, not significant) and diverge in informative ways: Igbo, despite low training-data resourcedness, achieves the best non-English cluster separation, while Arabic achieves the highest directional alignment with English yet only middling separation. This dissociation offers a candidate explanation for previously unexplained cross-lingual steering failures: high alignment alone does not guarantee that a language's internal representation is cleanly organized enough for a transplanted direction to act reliably. We release our full extraction and analysis pipeline and discuss concrete next steps for isolating the mechanism behind Arabic's specific vulnerability.
Chat is not available.
Successful Page Load