Zero-shot neural predictivity in human prefrontal cortex with a massively multi-task multimodal transformer

Working memory supports a broad spectrum of behaviors and higher cognitive abilities, with the prefrontal cortex playing a central role in this capacity. Although prior work has identified which brain regions are engaged in specific working memory tasks, and in some cases how they contribute, we still lack a general framework that can predict which regions will be recruited in novel tasks, what information they represent, and the computations they perform. To address this gap, we trained a single neural network on millions of visual decision-making tasks with sensory-realistic inputs, aiming to build a generalized model of working memory. We evaluated the model against an fMRI dataset spanning 12 tasks and hundreds of distinct conditions, testing its ability to capture neural activity across the brain, with a focus on the prefrontal cortex. Our results show that large models trained on a broad distribution of tasks can predict brain activity zero-shot, outperforming even models trained directly on the target tasks. This ability improves further with model size, which consistently enhances prediction accuracy. Furthermore, analyses of layer-to-region correspondences largely conformed with the theories of hierarchical organization along the rostro-caudal axis of the prefrontal cortex. These findings suggest that neural network models hold significant potential not only for simulating neural activity in regions previously difficult to model, but also for revealing how the brain encodes, organizes, and manipulates task information during working memory.