Timezone: »
Shannon's entropy is a basic quantity in information theory, and a fundamental building block for the analysis of neural codes. Estimating the entropy of a discrete distribution from samples is an important and difficult problem that has received considerable attention in statistics and theoretical neuroscience. However, neural responses have characteristic statistical structure that generic entropy estimators fail to exploit. For example, existing Bayesian entropy estimators make the naive assumption that all spike words are equally likely a priori, which makes for an inefficient allocation of prior probability mass in cases where spikes are sparse. Here we develop Bayesian estimators for the entropy of binary spike trains using priors designed to flexibly exploit the statistical structure of simultaneouslyrecorded spike responses. We define two prior distributions over spike words using mixtures of Dirichlet distributions centered on simple parametric models. The parametric model captures highlevel statistical features of the data, such as the average spike count in a spike word, which allows the posterior over entropy to concentrate more rapidly than with standard estimators (e.g., in cases where the probability of spiking differs strongly from 0.5). Conversely, the Dirichlet distributions assign prior mass to distributions far from the parametric model, ensuring consistent estimates for arbitrary distributions. We devise a compact representation of the data and prior that allow for computationally efficient implementations of Bayesian least squares and empirical Bayes entropy estimators with large numbers of neurons. We apply these estimators to simulated and real neural data and show that they substantially outperform traditional methods.
Author Information
Evan Archer (Sony AI)
Il Memming Park (Stony Brook University)
Jonathan W Pillow (UT Austin)
Jonathan Pillow is an assistant professor in Psychology and Neurobiology at the University of Texas at Austin. He graduated from the University of Arizona in 1997 with a degree in mathematics and philosophy, and was a U.S. Fulbright fellow in Morocco in 1998. He received his Ph.D. in neuroscience from NYU in 2005, and was a Royal Society postdoctoral reserach fellow at the Gatsby Computational Neuroscience Unit, UCL from 2005 to 2008. His recent work involves statistical methods for understanding the neural code in single neurons and neural populations, and his lab conducts psychophysical experiments designed to test Bayesian models of human sensory perception.
Related Events (a corresponding poster, oral, or spotlight)

2013 Spotlight: Bayesian entropy estimation for binary spike train data using parametric prior knowledge »
Fri. Dec 6th 11:30  11:34 PM Room Harvey's Convention Center Floor, CC
More from the Same Authors

2021 : Neural Latents Benchmark ‘21: Evaluating latent variable models of neural population activity »
Felix Pei · Joel Ye · David Zoltowski · Anqi Wu · Raeed Chowdhury · Hansem Sohn · Joseph O'Doherty · Krishna V Shenoy · Matthew Kaufman · Mark Churchland · Mehrdad Jazayeri · Lee Miller · Jonathan Pillow · Il Memming Park · Eva Dyer · Chethan Pandarinath 
2022 Poster: Value Function Decomposition for Iterative Design of Reinforcement Learning Agents »
James MacGlashan · Evan Archer · Alisa Devlic · Takuma Seno · Craig Sherstan · Peter Wurman · Peter Stone 
2017 Spotlight: Fast amortized inference of neural activity from calcium imaging data with variational autoencoders »
Artur Speiser · Jinyao Yan · Evan Archer · Lars Buesing · Srinivas C Turaga · Jakob H Macke 
2017 Poster: Fast amortized inference of neural activity from calcium imaging data with variational autoencoders »
Artur Speiser · Jinyao Yan · Evan Archer · Lars Buesing · Srinivas C Turaga · Jakob H Macke 
2016 : Jonathan Pillow : Scalable Inference for Structured Hierarchical Receptive Field Models »
Jonathan W Pillow 
2016 Poster: Linear dynamical neural population models through nonlinear embeddings »
Yuanjun Gao · Evan Archer · Liam Paninski · John Cunningham 
2014 Workshop: Large scale optical physiology: From dataacquisition to models of neural coding »
Il Memming Park · Jakob H Macke · Ferran Diego Andilla · Eftychios Pnevmatikakis · Jeremy Freeman 
2014 Poster: Optimal priordependent neural population codes under shared input noise »
Agnieszka GrabskaBarwinska · Jonathan W Pillow 
2014 Poster: Inferring sparse representations of continuous signals with continuous orthogonal matching pursuit »
Karin C Knudson · Jacob Yates · Alexander Huk · Jonathan W Pillow 
2014 Poster: Inferring synaptic conductances from spike trains with a biophysically inspired point process model »
Kenneth W Latimer · E.J. Chichilnisky · Fred Rieke · Jonathan W Pillow 
2014 Poster: Lowdimensional models of neural population activity in sensory cortical circuits »
Evan Archer · Urs Koster · Jonathan W Pillow · Jakob H Macke 
2014 Poster: Sparse Bayesian structure learning with dependent relevance determination prior »
Anqi Wu · Mijung Park · Sanmi Koyejo · Jonathan W Pillow 
2013 Poster: Spike train entropyrate estimation using hierarchical Dirichlet process priors »
Karin C Knudson · Jonathan W Pillow 
2013 Poster: Universal models for binary spike patterns using centered Dirichlet processes »
Il Memming Park · Evan Archer · Kenneth W Latimer · Jonathan W Pillow 
2013 Poster: Spectral methods for neural characterization using generalized quadratic models »
Il Memming Park · Evan Archer · Nicholas Priebe · Jonathan W Pillow 
2013 Poster: Bayesian inference for low rank spatiotemporal neural receptive fields »
Mijung Park · Jonathan W Pillow 
2012 Poster: Fully Bayesian inference for neural models with negativebinomial spiking »
Jonathan W Pillow · James Scott 
2012 Poster: Bayesian active learning with localized priors for fast receptive field characterization »
Mijung Park · Jonathan W Pillow 
2012 Poster: Bayesian estimation of discrete entropy with mixtures of stickbreaking priors »
Evan Archer · Jonathan W Pillow · Il Memming Park 
2011 Session: Oral Session 13 »
Jonathan W Pillow 
2011 Poster: Bayesian SpikeTriggered Covariance Analysis »
Il Memming Park · Jonathan W Pillow 
2011 Poster: Active learning of neural response functions with Gaussian processes »
Mijung Park · Greg Horwitz · Jonathan W Pillow 
2011 Spotlight: Active learning of neural response functions with Gaussian processes »
Mijung Park · Greg Horwitz · Jonathan W Pillow 
2011 Tutorial: Flexible, Multivariate Point Process Models for Unlocking the Neural Code »
Jonathan W Pillow 
2010 Poster: A novel family of nonparametric cumulative based divergences for point processes »
Sohan Seth · Il Memming Park · Austin J Brockmeier · Mulugeta Semework · John S Choi · Joseph T Francis · Jose C Principe 
2009 Oral: Timerescaling Methods for the Estimation and Assessment of NonPoisson Neural Encoding Models »
Jonathan W Pillow 
2009 Poster: Timerescaling methods for the estimation and assessment of nonPoisson neural encoding models »
Jonathan W Pillow 
2008 Poster: Characterizing neural dependencies with Poisson copula models »
Pietro Berkes · Frank Wood · Jonathan W Pillow 
2008 Spotlight: Characterizing neural dependencies with Poisson copula models »
Pietro Berkes · Frank Wood · Jonathan W Pillow 
2007 Oral: Neural characterization in partially observed populations of spiking neurons »
Jonathan W Pillow · Peter E Latham 
2007 Poster: Neural characterization in partially observed populations of spiking neurons »
Jonathan W Pillow · Peter E Latham