Wiki data to Wikipedia article; Evaluation of gender bias in data for LLM training
Abstract
Wiki data to Wikipedia article; Evaluation of gender bias in data for LLM training Large language models often fail on questions where their training data are sparse. Information about African women is particularly scarce online compared to that about men. Because Wikipedia contributes to the mix of data used to pre‑train LLMs, we trace how representation drops across its four‑stage pipeline: (1) a person is recorded in Wikidata, (2) an article is or isn’t created about them, (3) that article is or isn’t retrieved for a query, and (4) an LLM answers using what it has learned or can find online. We show that the failure occurs entirely at stage 2, the point where articles are created. Using SPARQL over Wikidata for every African country, we measure the female-to-male ratio of people who have a Wikipedia article in any language, with bootstrap confidence intervals from multinomial resampling. The majority of countries show a gap whose entire interval lies below parity, with the worst near 0.4. Women appear in Wikidata at rates near parity with men across most of the continent, but only reach a Wikipedia article at roughly half that rate in the worst-affected countries ,about 20 women for every 50 men. To further test the quality of articles that are available on wikipedia, we examine the gendered differences in article lengths and accuracy in LLM answers for. First, we sample 250 articles per country and language edition with hash-based random sampling (We use hash-based random sampling to avoid the bias of naive LIMIT queries, which tend to select older, longer, male-authored pages), we find women's articles are longer than men's in four of five editions, significant after multiple-testing correction. Second, we test retrieval using two standard dense retrievers (intfloat/e5 and BAAI/bge) to match each person's Wikidata description against Wikipedia article openings, checking where the correct article ranks among ~800 candidates per country-encoder cell and comparing mean rank by gender. After multiple-testing correction, 11 of 12 country-language cells show no significant difference; in the one that does, women's articles rank higher than men's. The bias is not in how women are described once an article exists, nor in whether that description is retrieved, it is in whether an article gets created at all.