Dialogic Deference: LLMs Can't Judge Who's Right in a Conversation

LLMs are increasingly deployed as third-party judges, yet their reliability when evaluating speakers in dialogue remains poorly understood. We show that LLMs judge identical claims differently depending on framing: the same content receives different verdicts when presented as a statement to verify ("Is this statement correct?") versus attributed to a speaker ("Is this speaker correct?"). We call this dialogic deference and introduce DialDefer, a framework for detecting and mitigating these framing-induced judgment shifts. Our Dialogic Deference Score (DDS) captures directional shifts that aggregate accuracy obscures. Across ten domains, 3k+ instances, and five models, conversational framing induces statistically significant deference ($p < .0001$) while accuracy remains stable ($<$2 pp), with effects amplifying 2--5$\times$ on naturalistic Reddit conversations. Effects are domain-dependent: a single model shifts toward deference on advice domains and toward skepticism on technical ones. Speaker-label ablations reveal that human-vs-LLM attribution drives the largest shifts (17.7 pp swing). A taxonomy of 2k+ judgment flips identifies Social Framing (27%) as the primary driver. Our proposed mitigation reduces deference without over-correction, but no single intervention eliminates it, revealing a calibration problem beyond accuracy optimization.