Better Representations, Better Speech BCIs: a Multitask Approach

Speech brain-computer interfaces represent a critical frontier for restoring communication in individuals with locked-in syndromes. While current approaches focus on scaling datasets and post-hoc language modeling, these strategies face immediate constraints due to data scarcity in neural recording settings. We propose a complementary path: enhancing representation learning through multitask training aligned with cortical encoding structure. Through systematic analysis of the largest intracortical speech dataset, we identify that acoustic features are most reliably encoded in speech motor cortex. Leveraging these insights, we develop multitask strategies combining phoneme prediction with acoustic regression and semantic objectives, achieving state-of-the-art performance: 13.7% word error rate. We also introduce the first end-to-end neural-to-sentence decoder for quasi-online communication. These results demonstrate that representation-first strategies substantially improve neural decoding within current data constraints while remaining compatible with future scaling efforts.