Demographic (Mis)Alignment of LLMs' Perception of Offensiveness

The assessment of offensive language varies across demographic groups. As large language models (LLMs) increasingly replace human annotators, concerns arise about potential demographic biases in LLM-generated annotations. This study extends research on LLM sociodemographic biases by testing their alignment with human judgments on offensive language across three datasets (POPQUORN, Social Bias Frames, UC Berkeley's Measuring Hate Speech) using three LLMs (GPT-4-Turbo, Mistral-7B-Instruct, and Solar-10.7B-Instruct).We employed a scoring system mirroring human annotator options and adopted a Chain of Thoughts prompting strategy. We validated our findings through permutation tests, which showed significant alignment in all cases. Our results demonstrate that LLM-generated annotations align well with judgments produced by human annotators, though the degree of alignment varied across different gender, racial, and gender-racial groups. While permutation tests confirmed significant individual correlations, they did not address the significance of differences between these correlations. To rigorously assess these differences and account for sampling variability, we employed bootstrapping analysis, which revealed mostly non-significant differences between demographic alignments. Nevertheless, we observed some systematic biases, particularly between Black and White demographics.Overall, our research provides a clearer understanding of LLMs' capabilities and limitations in annotating offensive language. With the insights from this study, we are now better positioned to anticipate the outcomes of deploying these models for large-scale data annotation tasks.