Spoken Dialogue Arena: A Live, Full-Duplex A/B Evaluation Platform for Spoken Dialogue Systems

We present Spoken Dialogue Arena, an open browser-based platform for live A/B evaluation of full-duplex spoken dialogue systems. Unlike conventional static evaluation using pre-recorded audio, the platform lets users converse with two anonymized systems in turn, rate them along four dimensions grounded in ITU-T speech and dialogue quality standards, and cast a pairwise preference vote. This live protocol directly captures interactional qualities such as response timing, barge-in, and backchannels, which are central to full-duplex dialogue. System order is counterbalanced, pairwise votes are aggregated using a Bradley-Terry model, and recording is optional and separated from participation. We describe the platform and evaluation protocol and report results from a ten-week internal pilot as a preliminary platform evaluation rather than a model leaderboard. Within the pilot, both native full-duplex systems compared directly against the commercial anchor showed a wide gap in audio quality and response content that narrowed to near parity in speech timing and conversational manner.