Tracing Whose Voices Survive AI Summarization: An Evidence-Linked Audit Interface for Civic Synthesis
Abstract
Language-model summaries can make a contested civic record appear more settled than the underlying contributions. Qualifiers, minority positions, local consequences, and unresolved disagreement may disappear even when a summary remains fluent and factually plausible. This paper presents Provenance Lab, an open-source visual analytics application that helps facilitators examine whose contributions remain traceable in civic synthesis. It links each summary sentence and claim back to candidate source passages, keeps weak, missing, and unassessed cases visible instead of folding them into an aggregate score, separates recorded system history from inferred semantic relationships, and carries the evidence through revision, participant response, and export. The empirical assessment tests the evidence-retrieval component on meeting, civic-minute, and policy-argument datasets that carry external reference links. A fixed semantic-search model ranked linked passages more accurately than a standard keyword baseline on all three, although an annotated evidence passage appeared first in only 55.9% of eligible meeting queries, which is why the interface offers ranked candidates for inspection rather than a single answer. A further analysis of published deliberation data found that how well a summary represented an individual participant helped predict that participant's stated preference, while summary-level dispersion statistics including the Gini coefficient added no detectable information once individual representation was included. This argues against treating aggregate inequality scores, the quantities an audit interface is most likely to display prominently, as self-validating fairness measures. These results support the interface as a way to locate candidate evidence and expose possible representational loss. They do not establish fairness, causal influence, or democratic legitimacy.