Creating Lexical Resources for Moroccan Darija: Transforming Print Dictionaries into Structured Digital Datasets
Abstract
Moroccan Darija, an under-resourced North African variety spoken by over 30 million people, lacks standardized digital lexicographical resources. While legacy print dictionaries contain rich vernacular and specialized domain terminology (e.g., maritime, architectural), digitizing them via commercial Optical Character Recognition (OCR) systems introduces severe Modern Standard Arabic (MSA) diglossic bias and character degradation. To bridge the gap between analog texts and machine-readable Natural Language Processing (NLP) infrastructure, we introduce a Human-in-the-Loop (HITL) methodology for lexical normalization, multi-source consolidation, and Linked Open Data (LOD) integration. Operating across six legacy print resources, our workflow: (1) extracts morphological base forms and establishes Unicode standardization for non-MSA phonemes (/g/, /p/, /v/); (2) performs inner polysemy disambiguation and outer cross-dictionary deduplication with source provenance tracking; and (3) maps consolidated entries to the Wikidata Lexeme schema using custom language-specific morphological templates. From 8,639 raw OCR records, our pipeline distilled 7,294 unique master Lexemes (a 15.6% consolidation rate). A stratified manual quality audit (N=100) confirmed a 100% precision rate across base lemmatization and Part-of-Speech tagging. Benchmarking zero-shot LLM baselines (Gemini) on raw OCR text yielded a 25.0% error rate, primarily failing to normalize regional character variants (retaining raw OCR artifacts like Persian Gaf {گ} instead of standardized Darija Keheh {ݣ}) and plural forms. To date, we have published 696 master Lexemes, 9,805 morphological forms, and 69 provenance claims to Wikidata under a CC0 license, making Darija computationally queryable via SPARQL and interoperable with global Linguistic Linked Data architectures. Evaluation proofs, audit logs, and LLM baseline test samples are available anonymously for peer review.