Who’s the Impostor? Multi‑Agent Social Deduction for Evaluating LLM Social Reasoning
Xiang Fu
Abstract
We present The Impostor Game, a controlled social‑deduction benchmark for evaluating interactive social reasoning in large language models via multi‑agent play. In each four‑player game, three agents share a majority word $w_m$ and one agent receives a related impostor word $w_i$; agents describe their words and then vote to identify the impostor. Across 90{,}720 games (9 models, 5 modes), vote‑network position best explains realized influence: coalition efficiency increases with centrality ($r \approx 0.87$). Performance varies substantially (impostor win 27.8–69.0%), and recognition (detection accuracy) shows a positive cross‑model trend with outcomes ($r \approx 0.61$, $p=0.17$). Speaking order is randomized; a pseudo‑arm ITT indicates that middle/late speaking modestly reduces impostor odds ($-1.11$ pp; 95% CI $[-1.64, -0.55]$), while seat‑index contrasts are descriptive. Despite having more information, team‑aware coordination underperforms team‑blind. Together, interaction signals—votes, outcomes, and the topology of the induced voting network—reveal limitations in social reasoning and coordination that are not captured by single‑agent evaluations.
Chat is not available.
Successful Page Load