Emotion-Aware Risk Modeling and Safety Evaluation of a Support-Oriented Language Model for High-Risk Mental Health Text

Support-oriented language models may encounter high-risk mental health language expressed directly or through euphemistic, third-person, joking, or fictional framing. This study presents an interpretable pipeline linking fine-grained emotion modeling, source-label classification, and structured safety stress testing of a locally deployed language model. A multilabel logistic-regression classifier trained on a noise-filtered GoEmotions corpus mapped text to 28 emotion probabilities derived from all-MiniLM-L6-v2 embeddings. The frozen emotion model was applied to 232,074 balanced Reddit posts from a public suicide/non-suicide corpus, and an L1-regularized logistic regression used only the 28 emotion probabilities to predict the source-derived label. The resulting coefficients informed a heuristic concern signal for a local Llama 3 chatbot served through Ollama, together with conservative pattern-based overrides and a hidden safety prompt. The chatbot was evaluated on 888 synthetic prompts spanning direct, euphemistic, fictional, hopelessness, third-person, joking, policy, and explicit testing formulations. On a held-out set of 46,415 Reddit posts, the emotion-only classifier achieved ROC-AUC = 0.9549 and PR-AUC = 0.9546. At a recall-oriented threshold of 0.30, precision was 0.8373, recall was 0.9373, and F1 was 0.8845. Across chatbot trials, 594 responses (66.9%) were categorized as supportive/non-method and 209 (23.5%) as safe refusals; together, 803 responses (90.4%) fell into categories without an apparent method-like flag. Manual review found no explicit procedural self-harm instructions among flagged outputs, suggesting conservative over-calling by the automated rubric. The framework provides a reproducible baseline for auditing sensitive-language failure modes, but it does not establish clinical validity or real-world chatbot safety.