Demo: Real-time visualization and inference of Kernel Flow TD-NIRS data using the NVIDIA Holoscan SDK and Jetson Thor

While there is a plethora of technologies for measuring the electrical signals of the brain and body, few efforts have focused on improving scalable collection of hemodynamic brain activity maps. At Kernel, we’ve combined a unique optical measurement technology, time-domain functional near-infrared spectroscopy (TD-fNIRS), with traditional electroencephalography (EEG). This combination of technologies produces rich multimodal perspectives of brain function. The Kernel Flow headset also has an easy-to-use helmet-like form factor and streamlined setup procedures to enable data collection with less than 3 minutes of setup time in almost any setting. It is also one of the highest brain data bandwidth products on the market, producing over 100Mbps of raw data derived from thousands of measurement points over the whole head.

With this new technology and high data bandwidth come new challenges, like running real-time decoding algorithms using the full resolution of our technology. Kernel has partnered with NVIDIA to develop a solution combining Kernel’s multimodal datastream with NVIDIA’s Holoscan SDK for real-time AI applications running on the new NVIDIA Jetson Thor. This combined demonstration showcases the ability to perform real-time inference in real-world settings using commercially available edge devices. We have trained a simple model that decodes the block type (language vs. noise) from an auditory task dataset that was previously published (https://www.nature.com/articles/s41598-024-68555-9). In this interactive demo, attendees can wear the Kernel Flow headset and experience the classification model output in real time. In addition, they will be able to view real-time representations of the data streamed from the Kernel Flow device, observe workloads on the NVIDIA Jetson Thor device, and watch visualizations of the incoming data and model outputs. All code and training data will be made available through NVIDIA’s HoloHub platform (https://nvidia-holoscan.github.io/holohub/) after the NeurIPS workshop. This will be the first time this combination is shared publicly.

Attendees of the workshop will take away from the demo: - Hands-on experience using the Kernel Flow device to measure brain function. - An understanding of the HoloScan SDK and how it can be leveraged to perform real-time inference on high-bandwidth multi-modal neural data.