Born with a SilverSpoon? Investigating Socioeconomic Bias in LLMs

Socioeconomic bias shapes access to opportunity and perpetuates systemic inequality, yet its presence in large language models (LLMs) remains underexplored. We introduce \textsc{SILVERSPOON}, a 12{,}000-sample dataset designed to evaluate socioeconomic bias in LLMs across three dimensions: (1) \textbf{normative judgment evaluation} of underprivileged individuals in ethical dilemmas, (2) \textbf{demographic-driven profession prediction} combining race, gender, and location, and (3) \textbf{contextual narrative analysis} of generated stories. Using \textsc{SILVERSPOON}, we evaluate several state-of-the-art LLMs (GPT-4o, Llama3, Gemma, Alpaca, Zephyr, Yi) through quantitative metrics and qualitative analysis. Our results show that LLM outputs often fail to align with judgments favoring socioeconomically underprivileged individuals and instead amplify stereotypes linked to race and gender. In profession prediction, models disproportionately assign high-income roles to White and Indian men while associating Black and Hispanic individuals with low-income jobs. Narrative analysis further reveals subtle negative sentiment toward minority groups, particularly Hispanic and Black women. By releasing \textsc{SILVERSPOON} under a CC-BY 4.0 license, we aim to enable reproducible research on alignment and provide a foundation for mitigating socioeconomic bias in LLMs.