SovereignRAG-Eval: Compute- and Sovereignty-Aware Offline-to-Online Retrieval Evaluation

Retrieval-augmented generation is commonly evaluated with static English question sets, proprietary model judges, and mean effectiveness scores that omit energy, latency, and data movement. This evaluation stack is poorly matched to laboratories operating with limited accelerators, intermittent connectivity, multilingual queries, and restrictions on data export. We present SovereignRAG-Eval, a preregisterable protocol for evaluating retrieval application programming interfaces with locally executed Llama-family models while retaining human judgment and online behavior as validity anchors. The protocol links four evidence layers: pooled relevance judgments for offline retrieval; calibrated and perturbation-tested local judges for grounded answers; consented online interleaving for behavioral utility; and query-level resource and governance accounting. It reports site–language–task distributions and worst-stratum outcomes rather than a continental or global mean. A constrained selection rule exposes the quality–latency–energy frontier and treats data egress as a measurable system failure. We synthesize open-access research on retrieval evaluation, African-language resources, model judging, sustainable machine learning, and data sovereignty, then define hypotheses, baselines, admission gates, uncertainty estimates, ablations, and a minimum viable laboratory workflow. This paper specifies a study; it does not claim unobserved deployment results. The goal is an evaluation process that remains reproducible and decision-relevant under the conditions in which locally governed systems actually operate.