DecisionDiff: Versioned Change Control for AI-Influenced Conference Decisions
Abstract
AI policy in peer review is becoming heterogeneous by design. In 2026, major ML venues already differ in author consent, reviewer permissions, disclosure, sanctioned tools, and experimental use of LLM assistance. This diversity is valuable, but conference infrastructure still treats AI-influenced decisions as ordinary text fields with little machine-readable provenance. We propose DecisionDiff, a versioned change-control protocol for AI-influenced review, meta-review, and program-chair decisions. The core principle is that AI may propose a state change, but no consequential scholarly decision should change silently. Every AI-influenced “patch” records the prior decision state, policy profile, input snapshot, tool and model version, proposed change, human acceptance or override, rationale, and downstream effect. We first provide a descriptive audit of public 2026 policies from ICML, ICLR, and NeurIPS, showing that the field is converging on human responsibility while diverging on operational permissions. DecisionDiff converts those differences into machine-readable policy profiles rather than requiring one universal rule. We define policy-diff, model-diff, evidence-diff, and decision-diff events, and propose audit metrics for silent automation, override visibility, policy compliance, and reproducibility. The framework extends version-control logic to conference governance: venues should be able to answer not only what decision was made, but what changed, under which policy, by which tool, and who authorized the change.