Readout Fitting for Low-Data Turn-Taking Preference Prediction

Conversational agents need to predict when to remain silent, give a brief acknowledgment, or start speaking. Poor predictions after fine-tuning may reflect how a model's features are used, rather than a lack of useful information. We study this problem on the public DuplexGen annotations using Qwen3-4B. We hold the fine-tuned model's features fixed and refit the readout, the layer that converts them into action probabilities. Prediction improves, but simple scenario averages account for much of the gain. The features still add information beyond these averages and shuffled-feature controls. We compare frozen models with fine-tuning initialized from a fitted readout (LP-FT) on a common split. Adjusting their probabilities on validation data, a step called calibration, removes much of the apparent advantage of freezing. With more flexible calibration, the frozen readout improves KL divergence over LP-FT by 0.001547, with a 95% dialogue-bootstrap interval of [-0.000143, 0.003092]. This KL advantage is uncertain; Brier still favors freezing. Training only the readout does not reproduce the deterioration seen when it is trained together with the language model. These results support checking scenario averages, readout fitting and calibration before attributing poor predictions to model features. The comparison uses 300 test dialogues examined during development. It does not establish that freezing beats well-tuned adaptation; independent confirmation and live-interaction effects remain untested.