Readout Fitting for Low-Data Turn-Taking Preference Prediction: A Diagnostic Study

Conversational agents predict when to remain silent, give a brief acknowledgment, or take the floor. Poor predictions after fine-tuning need not mean that the model's features are uninformative. We diagnose this distinction on DuplexGen's text-based preference annotations using Qwen3-4B. Refitting a fixed adapted encoder's output layer reduces KL divergence from 0.3839 to 0.3088. A scenario-only predictor reaches 0.3291, while aligned features improve on scenario priors and shuffled-feature controls. In a separate comparison on a shared training/validation split, calibration removes much of freezing's apparent advantage over fine-tuning. With affine calibration, the frozen model's KL gain over fine-tuning initialized from a fitted readout is 0.0015, with a 95% dialogue-bootstrap interval of [-0.0001, 0.0031]; Brier score still favors freezing. These results support checking simple priors, readout fitting and calibration before attributing poor predictions to the encoder. This exploratory study uses 300 test dialogues examined during development. It establishes neither superiority over well-tuned adaptation nor better live turn-taking behavior.