Timezone: »

Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice
Rylan Schaeffer · Mikail Khona · Leenoy Meshulam · Brain Laboratory International · Ila Fiete

Wed Dec 09 09:00 PM -- 11:00 PM (PST) @ Poster Session 4 #1264

We study how recurrent neural networks (RNNs) solve a hierarchical inference task involving two latent variables and disparate timescales separated by 1-2 orders of magnitude. The task is of interest to the International Brain Laboratory, a global collaboration of experimental and theoretical neuroscientists studying how the mammalian brain generates behavior. We make four discoveries. First, RNNs learn behavior that is quantitatively similar to ideal Bayesian baselines. Second, RNNs perform inference by learning a two-dimensional subspace defining beliefs about the latent variables. Third, the geometry of RNN dynamics reflects an induced coupling between the two separate inference processes necessary to solve the task. Fourth, we perform model compression through a novel form of knowledge distillation on hidden representations -- Representations and Dynamics Distillation (RADD)-- to reduce the RNN dynamics to a low-dimensional, highly interpretable model. This technique promises a useful tool for interpretability of high dimensional nonlinear dynamical systems. Altogether, this work yields predictions to guide exploration and analysis of mouse neural data and circuity.

Author Information

Rylan Schaeffer (Harvard University)
Mikail Khona (MIT)
Leenoy Meshulam (Massachusetts Institute of Technology MIT)
Brain Laboratory International (International Brain Laboratory)
Ila Fiete (Massachusetts Institute of Technology)

More from the Same Authors