When Local Safety Does Not Compose: Measuring the Limits of Automated Risk Assessment for Cross-Institutional Agentic Systems

As risk assessment of agentic artificial intelligence (AI) shifts from human analysts to AI assistants, a basic measurement is missing: can AI systems reliably apply a risk codebook at all? We study this question where compositional risk is most acute — agentic deployments that cross institutional boundaries — using multi-institutional science as a stress test: planning agents at one facility drive high-performance computing workflows at another, federated learning couples model state across laboratories, and autonomous experiments actuate physical instruments. Safety guarantees established for individual components do not automatically compose in these settings, and we operationalise the resulting risks as ten campaign-level motifs. We then measure codebook execution with a frozen protocol: nine annotation runs from three developers over a 24-case corpus, AI adjudication of all 123 resulting disagreements, and an independent expert anchor. The headlineis simple: AI systems can reliably describe compositional risk, but cannot yet reliably judge whether existing governance frameworks adequately cover it —descriptive fields reach α = 0.46–0.74 while coverage adequacy sits at chance level. The judgment becomes substantially more reliable when evidence is restructured for adjudication (α = 0.18 → 0.60, non-overlapping intervals), and the residual unreliability decomposes into codebook underdetermination and model divergence, which matter differently for deployment. Against the expert anchor, automated adjudication matches expert judgment on 63% of contested rows (47–77% by system) and errs in one direction: AI adjudicators are more likely than the expert to treat a composed risk as already covered — optimistic about framework sufficiency precisely where optimism silently converts uncovered risks into “already governed” ones. AI can triage and characterise agentic risk; humans must still own coverage24 adequacy.