SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets

Scalar reward is insufficient for evaluating autonomous economic agents whose decisions affect physical resources. We introduce SolarChain-Eval, a benchmark for decentralized peer-to-peer solar markets that couples reinforcement-learning policies to a physics oracle. An evaluation-only LLM Planner/Auditor provides bounded oversight of proposed actions. The protocol relates episode utility to physical invalidity and its market consequences. Separate diagnostics characterize execution friction and temporal actions; city-level allocation and oversight traces make distribution and intervention visible. Across three archived seeds, 90 rollouts per policy show a utility--safety trade-off: removing the physics penalty increases apparent return while increasing artificial liquidity for learned policies. Planner/Auditor interventions make selected corrections traceable, but do not compensate for reward misspecification. The accompanying artifact package preserves the archived inputs, checkpoints, logs, source snapshot, and scripts needed to reproduce the reported summaries.