Untangling Under Noise: Distinct Manifold Regimes in Human Cortex and Speech-Language Models
Francis Pingfan Chien ⋅ Yu-En Tsai ⋅ Chi-Hsiang Chao ⋅ Po-Jang Hsieh ⋅ Yu Tsao
Abstract
Speech perception deteriorates in noise, but behavioral loss does not reveal how population representations are reorganized. We combined fMRI, linear decoding, and mean-field theoretic manifold analysis (MFTMA) to compare Clean and noise-degraded Mandarin speech in 25 listeners and across seven speech-language models. Although intelligibility and comprehension declined, Clean versus Noisy speech remained decodable throughout 20 bilateral auditory-language regions. Manifold geometry changed much more selectively: Noisy cortical responses showed lower manifold dimension $D_{\mathrm{M}}$ in parietal, frontal, and sensorimotor regions, broadly stable manifold radius $R_{\mathrm{M}}$, and local increases in the MFTMA-derived capacity descriptor $\alpha_{\mathrm{MFT}}$. Artificial systems showed no common response to degradation. WavLM exhibited compact compression; Whisper encoders combined dimensional compression with radial expansion; final-token states from Whisper decoders and GPT-2 XL expanded in both radius and dimension; and Phi-4-mini-instruct paired strong radial expansion with dimensional collapse. Thus, perceptual difficulty, linear decodability, and manifold geometry capture distinct consequences of acoustic degradation. Noise reorganizes representations through region-, architecture-, and stage-dependent geometric regimes rather than a universal form of tangling.
Chat is not available.
Successful Page Load