When Does a Speech Model Know to Hand Off? Reading Competence from Hidden States for Real-Time Escalation

A full-duplex speech model must choose whether to answer immediately or wait for stronger reasoning. Recent systems learn related decisions through special- token generation within the dialogue model. We study a post-training alternative: freeze MiniCPM-o 4.5 and fit a logistic failure readout on hidden states at its native listen-to-speak transition. The gate adds no model generation, and its thresholds can be retuned without changing the duplex policy. Text-trained probes also read spoken questions, including unseen human recordings. On a 240-query internal benchmark, selective handoff improves answer-content accuracy by 27.1 points over local answering and reduces mean server time to first answer audio by 31.5% relative to always escalating. It outperforms matched random routing across exter- nal speech-QA pools and a zero-shot GPT-5.4 mini transcript router in accuracy. Refitting the recipe on a second duplex model family improves routing in cached replay. These results support adding competence-aware expert access to an exist- ing speech agent without further duplex training.