Continuous Ethiopian Sign Language Recognition: Video and Skeleton-Based Models with Cross-Lingual Transfer

Sign language is an important means of communication for Deaf communities, yet most sign language recognition research focuses on a small number of well-resourced languages. Ethiopian Sign Language (EthSL) remains largely underrepresented, and existing studies have mainly considered isolated signs rather than continuous sentence-level recognition. This limits the development of assistive technologies that could improve communication, education, and access to information for Deaf individuals. To address this gap, we introduce a new multi-signer dataset for continuous Ethiopian Sign Language recognition. The dataset contains 2,220 video sequences covering 50 unique sentences performed by 22 signers. Using this dataset, we study two complementary approaches for continuous sign language recognition. The first is a video-based model that learns visual and motion information directly from video frames. The second is a lightweight skeleton-based model that uses hand and body keypoints extracted from videos to capture signer movements more efficiently. We also investigate whether knowledge learned from another sign language can help improve recognition in low-resource settings. Since large pretrained models for African sign languages are not available, we evaluate cross-lingual transfer using a frozen hand encoder trained on Saudi Sign Language. This allows us to examine whether representations learned from one sign language can benefit another with limited training data. Our experiments evaluate performance under both signer-independent and unseen-sentence settings. The video-based model achieves word error rates (WER) of 8.82\% and 58.0\%, respectively. The skeleton-based baseline achieves 18.5\% WER on signer-independent evaluation and 79.4\% WER on unseen sentences. After incorporating the pretrained Saudi Sign Language encoder, performance improves substantially to 8.2\% and 60.4\% WER, corresponding to absolute gains of 10.3 and 19.0 percentage points. These results show that cross-lingual transfer can significantly improve recognition performance, especially in low-resource sign language settings. Overall, our findings demonstrate that both video and skeletal representations can effectively support continuous Ethiopian Sign Language recognition, while cross-lingual transfer provides a practical solution for overcoming data scarcity. Beyond Ethiopian Sign Language, this work highlights the potential of transfer learning for other underrepresented sign languages and provides a foundation to build more accessible and inclusive sign language technologies for Deaf communities worldwide.