Anytime-Valid Federated Conformal RAG for LLM Swarms
Prasanjit Dubey ⋅ Xiaoming Huo
Abstract
Federated Conformal Retrieval-Augmented Generation (FC-RAG) gives answer-set coverage for a bandwidth-constrained shared large language model (LLM) scorer with decentralized retrievers, but a fixed-state certificate alone does not cover repeated inspection or history-selected refreshes. We introduce certified adaptive refreshes for Anytime-FC-RAG: select a score state, calibrate it on fresh data, charge every certificate attempt to one trajectory-wide calibration budget $\delta_{\text{cal}}$, and continue one betting wealth process $E_t$ across deployments. If the active certificate bounds next-query conditional miss risk by a pre-query center $b_t$, then, for an evidence budget $\delta_e$, $\mathbb{P}(\sup_t E_t\ge 1/\delta_e)\le \delta_e+\delta_{\text{cal}}$ despite history-dependent, certificate-preserving refreshes. The exact order-statistic certificate is finite-sample and distribution-free under the stated conditional score law. In an independent 100-segment model, the any-certificate-failure probability is $.9941$ without allocation and $.0488/.0490$ under Bonferroni/summable allocations; both schemes guarantee at most $.05$ without independence. With 100 candidate states, buffer reuse gave mean miscoverage $.1592$, versus $.0993$ after fresh calibration. In a fixed-state Qwen2.5-1.5B/MMLU-Pro replay, all 210 paths from 35 independently audited harmful cells alarmed by $5{,}000$ queries, with no alarms on 192 compatible or 30 no-drift paths. The resulting contract makes adaptive federated-RAG monitoring auditable under one trajectory-level false-alarm budget.
Chat is not available.
Successful Page Load