Demo: An Open-Source Ecosystem for Models of Multi-Modal Brain and Body Data

Overview torch_brain, brainsets, and temporaldata (https://torchbrain.org/) form an open-source ecosystem designed to make the modern deep learning toolbox and neuro-foundation models accessible for both neuroscientists and ML practitioners. These tools efficiently scale models to large datasets (TB-scale data), make neural data easier to use by providing access to hundreds of ready-for-training datasets across multiple brain modalities, and unlock new capabilities in neurotech. We believe this engineering layer is critical for building and adopting foundation models.

Why this matters Progress in language modeling was in large part made possible thanks to major engineering efforts put into making high performance training and inference pipelines accessible, and we are replicating that in neuroscience. Our goal is to lower the barriers to using powerful models, standardize access to datasets, and accelerate research. Already, we have seen the tools we’ve developed catalyze research across multiple groups.

The demo Our goal is to show attendees how these tools make cutting-edge methods approachable and usable in real-world research workflows. Specifically, the demo will highlight how researchers can: 1) Rapidly fine-tune pre-trained models on their own datasets. 2) Leverage standardized access to diverse datasets covering a wide range of neural modalities (EPhys, OPhys, EEG, iEEG). 3) Explore the use of neural embeddings to gain scientific insights.

Concretely, we will walk attendees through interactive notebooks showing how to fine-tune off-the-shelf models on new data and visualize neural embeddings. Our booth will also act as a help desk, where attendees can share their use cases, and we help them discover how they can use our open-source tools in their own workflows to streamline data access, train and fine-tune models, and ultimately accelerate neurotech and scientific discovery in neuroscience.