Evaluating Mental Health LLM Responses to Localized African English

Large language models are increasingly a first point of contact for people in psychological distress in low-resource settings, where users often write in localized English varieties rather than the standardized English on which safety evaluations are built. We test whether the quality of a mental health model's responses is sensitive to surface linguistic form when semantic intent is held constant. From MentalChat16K we sample 100 clinician-grounded prompts and construct two controlled variants of each: a heavily non-standard broken/pidgin register, and a lighter Ghanaian English register carrying Twi lexical influence. The resulting 300 prompts are answered by MentaLLaMA-chat-7B and scored 1-5 against the dataset's reference response by an LLM judge. Composite quality falls from 3.94 under standard English to 3.79 under pidgin (p = 0.0027, Cohen's d = 0.31), driven by relevance (4.03 to 3.74, p = 0.0072); the Ghanaian condition is statistically indistinguishable from standard English on every metric. Inspecting the worst-affected scenarios shows the mean shift understates the tail: identical concerns, reworded, turned supportive replies into outright refusals, generic boilerplate, or corrective instruction. Safety scored 5/5 on all 300 outputs, which we read as an insensitive rubric rather than as evidence of uniform safety. The degradation is narrow but real and concentrated where aggregate scores hide it.