Timezone: »
Neuroscience experiments often require training animals to perform tasks designed to elicit various sensory, cognitive, and motor behaviors. Training typically involves a series of gradual adjustments of stimulus conditions and rewards in order to bring about learning. However, training protocols are usually hand-designed, relying on a combination of intuition, guesswork, and trial-and-error, and often require weeks or months to achieve a desired level of task performance. Here we combine ideas from reinforcement learning and adaptive optimal experimental design to formulate methods for adaptive optimal training of animal behavior. Our work addresses two intriguing problems at once: first, it seeks to infer the learning rules underlying an animal's behavioral changes during training; second, it seeks to exploit these rules to select stimuli that will maximize the rate of learning toward a desired objective. We develop and test these methods using data collected from rats during training on a two-interval sensory discrimination task. We show that we can accurately infer the parameters of a policy-gradient-based learning algorithm that describes how the animal's internal model of the task evolves over the course of training. We then formulate a theory for optimal training, which involves selecting sequences of stimuli that will drive the animal's internal policy toward a desired location in the parameter space. Simulations show that our method can in theory provide a substantial speedup over standard training methods. We feel these results will hold considerable theoretical and practical implications both for researchers in reinforcement learning and for experimentalists seeking to train animals.
Author Information
Ji Hyun Bak (Princeton University)
Jung Choi
Ilana Witten
Athena Akrami (Princeton University)
Jonathan Pillow (Princeton University)
More from the Same Authors
-
2021 : Neural Latents Benchmark ‘21: Evaluating latent variable models of neural population activity »
Felix Pei · Joel Ye · David Zoltowski · Anqi Wu · Raeed Chowdhury · Hansem Sohn · Joseph O'Doherty · Krishna V Shenoy · Matthew Kaufman · Mark Churchland · Mehrdad Jazayeri · Lee Miller · Jonathan Pillow · Il Memming Park · Eva Dyer · Chethan Pandarinath -
2022 : Non-exchangeability in Infinite Switching Linear Dynamical Systems »
Victor Geadah · Jonathan Pillow -
2022 Poster: Dynamic Inverse Reinforcement Learning for Characterizing Animal Behavior »
Zoe Ashwood · Aditi Jha · Jonathan Pillow -
2022 Poster: Extracting computational mechanisms from neural data using low-rank RNNs »
Adrian Valente · Jonathan Pillow · Srdjan Ostojic -
2020 Poster: High-contrast “gaudy” images improve the training of deep neural network models of visual cortex »
Benjamin Cowley · Jonathan Pillow -
2020 Poster: Identifying signal and noise structure in neural population activity with Gaussian process factor models »
Stephen Keeley · Mikio Aoi · Yiyi Yu · Spencer Smith · Jonathan Pillow -
2020 Poster: Inferring learning rules from animal decision-making »
Zoe Ashwood · Nicholas Roy · Ji Hyun Bak · Jonathan Pillow -
2018 Poster: Scaling the Poisson GLM to massive neural datasets through polynomial approximations »
David Zoltowski · Jonathan Pillow -
2018 Poster: Efficient inference for time-varying behavior during learning »
Nicholas Roy · Ji Hyun Bak · Athena Akrami · Carlos Brody · Jonathan Pillow -
2018 Poster: Model-based targeted dimensionality reduction for neuronal population data »
Mikio Aoi · Jonathan Pillow -
2018 Poster: Power-law efficient neural codes provide general link between perceptual bias and discriminability »
Michael J Morais · Jonathan Pillow -
2018 Poster: Learning a latent manifold of odor representations from neural responses in piriform cortex »
Anqi Wu · Stan Pashkovski · Sandeep Datta · Jonathan Pillow -
2017 Poster: Gaussian process based nonlinear latent structure discovery in multivariate spike train data »
Anqi Wu · Nicholas Roy · Stephen Keeley · Jonathan Pillow -
2016 Poster: Bayesian latent structure discovery from multi-neuron recordings »
Scott Linderman · Ryan Adams · Jonathan Pillow -
2016 Poster: A Bayesian method for reducing bias in neural representational similarity analysis »
Mingbo Cai · Nicolas W Schuck · Jonathan Pillow · Yael Niv -
2015 Poster: Convolutional spike-triggered covariance analysis for neural subunit models »
Anqi Wu · Il Memming Park · Jonathan Pillow