Comparing Multilingual ASR Models for Low-Resource African Languages: What Works, and When

Fine-tuned speech recognition models for low-resource African languages don't just differ in how many mistakes they make but also in what kind of mistakes. On three Bantu languages from WAXAL (Luganda, Shona, Lingala), a fine-tuned CTC model (MMS) gets more words right than fine-tuned Whisper, but almost never writes a comma. Whisper is the reverse, weaker on content but stronger on punctuation. Because punctuation is exactly where the two models disagree, any fix that only trusts them where they already agree can't fix it. We compare three fixes, time-aligned confidence scoring, a WhisperX-style setup using MMS only for word timing, and a small mT5 model trained on the two models' real errors. The best combination gives a small but real overall gain. Separately, a much bigger foundation model, Omnilingual ASR, beats both fine-tuned MMS and fine-tuned Whisper on all three languages, by a wide margin on Shona and Lingala, but not unassisted. Its tokenizer can't represent punctuation or casing at all, so every one of its numbers already includes a separate restoration pass run afterward. We also tried adding a KenLM language model and GPT2 rescoring to MMS, the obvious next step, but it made MMS worse, not better, so we dropped both and kept plain CTC decoding. That leaves an open question, can we build a single model that is good at both words and structure, without needing an extra pass to fix one or the other?