From Linguistic Knowledge to Decodable Text: A Benchmark Ladder for Kiswahili Literacy

Early literacy depends on children reading text they can sound out using what they have been taught. In Kiswahili and many other languages used in low- and middle-income countries, suitable decodable text is much scarcer than in English. AI-generated text could help address this gap by producing material tailored to a learner’s instructional level. However, evaluation of large language models’ ability to generate curriculum-aligned decodable material remains limited, particularly for lower-resourced languages. We introduce a three-rung benchmark ladder to diagnose failures in decodable-text generation. Linguistic Knowledge probes orthography, phonology, morphology, syntax, and semantics. Decodability Assessment tests whether a model can identify words a child cannot yet decode given a specified instructional sequence. Decodable Text Generation tests whether models can produce coherent, engaging, age-appropriate, safe, and decodable text. In this work, we instantiate this framework for Kiswahili, grounding benchmark items in primary sources and developing a formal decodability specification aligned with Kenya’s Tusome curriculum in collaboration with literacy experts. We encoded these criteria in deterministic tooling that produces verifiable word-level labels.