Evaluating Automatic Speech Recognition (ASR) Systems on Yorùbá STEM Educational Content

Recent developments in Automatic Speech Recognition (ASR) have considerably enhanced speech technology for high-resource languages. Despite ongoing efforts in Automatic Speech Recognition for low-resource languages, a wide gap still remains between the technical performance of global models and the real-world linguistic needs of communities, particularly in domain-specific contexts such as Science, Technology, Engineering, and Mathematics (STEM). Existing speech models are typically trained on broad-domain datasets including news broadcasts, religious texts, and conversational speech, which do not adequately capture technical vocabulary or domain-specific speech patterns. This study evaluates the performance of four existing ASR systems MMS-1B-All, NCAIR1/Yoruba-ASR, Omilingual, and a fine-tuned Whisper Small model adapted for Yorùbá-English code-switching on a curated domain-specific Yorùbá STEM speech dataset. Using Word Error Rate (WER) and Tone Error Rate (TER) as evaluation metrics, the study identifies domain-specific limitations including the transcription of technical terminology, tonal diacritic errors, loanword adaptation, and code-switching between Yorùbá and English. All four models recorded high error rates, with Omilingual performing best at a WER of 83.13% and TER of 47.27%, followed by NCAIR1/Yoruba-ASR at 91.76% WER and 52.52% TER, MMS-1B-All at 93.58% WER and 56.91% TER, and Whisper Small at 97.42% WER and 49.14% TER. These findings establish baseline performance indicators for Yorùbá STEM ASR and highlight the urgent need for domain-specific datasets and models for African languages in STEM education in order to localise STEM subjects and bridge the linguistic gap in scientific communication.