Subword Tokenization for Low-Resource Amharic-to-Gedeo Neural Machine Translation: Translation Quality versus Morphological Fidelity

Ethiopia is home to a highly diverse linguistic landscape, yet digital language resources remain scarce for many of its languages. Amharic, the country’s federal working language, has benefited from substantial advances in natural language processing (NLP), whereas Gedeo, spoken by millions in southern Ethiopia, remains significantly under-resourced in the digital domain. The two languages are also typologically distinct: Amharic is a South Ethiosemitic language characterized by root-and-pattern morphology, while Gedeo belongs to the Highland East Cushitic branch and exhibits predominantly suffixing morphology and case marking. These linguistic differences make the choice of subword tokenization particularly important, as tokenization is a fundamental component of neural machine translation (NMT) systems and can substantially influence vocabulary coverage, sequence length, and downstream translation performance, especially for a low-resource and typologically diverse language pair. We study Amharic-to-Gedeo NMT using a 48,367-sentence parallel corpus drawn from religious texts and manually revised web-scraped data., split into 38,694 training and 9,673 validation pairs. We fine-tune mT5-small with low-rank adaptation (LoRA, r=64, alpha=128 on the query/value projections) under three tokenization regimes: mT5's pretrained multilingual SentencePiece BPE vocabulary (250,112 tokens, covering 101 languages including Amharic but not Gedeo), a monolingual Unigram tokenizer trained only on the Gedeo side of the corpus (32,000 tokens), and a bilingual Unigram tokenizer trained jointly on Amharic and Gedeo (32,000 tokens). We evaluate translation quality with chrF++ and, using a new 658-word gold-standard Gedeo morphological segmentation set built by a native-speaker language teacher and journalist, measure how closely each tokenizer's induced subword boundaries match true morpheme boundaries. The pretrained multilingual BPE tokenizer achieves both the best translation quality (chrF++ = 11.54) and, unexpectedly, the best morphological boundary F1 (0.1382), despite never having seen Gedeo during pretraining. The Gedeo-only Unigram tokenizer performs worst on both metrics (chrF++ = 4.38; boundary F1 = 0.1169), while adding Amharic data to the Unigram vocabulary substantially narrows the gap (chrF++ = 7.56; boundary F1 = 0.1240) without closing it. Precision and recall reveal contrasting failure modes: BPE over-segments Gedeo words, achieving relatively high boundary recall (0.32) but low precision (0.09), whereas the Unigram tokenizers under-segment, with low recall (0.14–0.16) but comparatively higher precision. Neither tokenizer family is therefore well aligned with Gedeo morphological structure. We interpret these results as evidence that, in a small-model, limited-data, parameter-efficient fine-tuning setting, the cost of replacing mT5’s pretrained vocabulary and associated representations can outweigh the benefits of improved morphological alignment. Introducing a new 32,000-token vocabulary requires the model to learn new token representations, whereas retaining the pretrained BPE vocabulary preserves mT5’s multilingual representations. This tension between morphological adequacy and translation performance echoes recent findings for other Ge’ez-script languages, where morphology-aware tokenization improved intrinsic segmentation metrics without consistently improving downstream translation quality. We release both custom tokenizers and the first gold-standard Gedeo morphological segmentation set to support future research on this extremely low-resource language pair.