Timezone: »

 
Poster
Estimating the Unique Information of Continuous Variables
Ari Pakman · Amin Nejatbakhsh · Dar Gilboa · Abdullah Makkeh · Luca Mazzucato · Michael Wibral · Elad Schneidman

Fri Dec 10 08:30 AM -- 10:00 AM (PST) @

The integration and transfer of information from multiple sources to multiple targets is a core motive of neural systems. The emerging field of partial information decomposition (PID) provides a novel information-theoretic lens into these mechanisms by identifying synergistic, redundant, and unique contributions to the mutual information between one and several variables. While many works have studied aspects of PID for Gaussian and discrete distributions, the case of general continuous distributions is still uncharted territory. In this work we present a method for estimating the unique information in continuous distributions, for the case of one versus two variables. Our method solves the associated optimization problem over the space of distributions with fixed bivariate marginals by combining copula decompositions and techniques developed to optimize variational autoencoders. We obtain excellent agreement with known analytic results for Gaussians, and illustrate the power of our new approach in several brain-inspired neural models. Our method is capable of recovering the effective connectivity of a chaotic network of rate neurons, and uncovers a complex trade-off between redundancy, synergy and unique information in recurrent networks trained to solve a generalized XOR~task.

Author Information

Ari Pakman (Columbia University)
Amin Nejatbakhsh (Columbia University)
Dar Gilboa (Columbia University)
Abdullah Makkeh (University of Goettingen)
Luca Mazzucato (University of Oregon)
Michael Wibral (Georg August University)
Elad Schneidman (Weizmann Institute of Science)

More from the Same Authors

  • 2021 : Sinkhorn EM: An Expectation-Maximization algorithm based on entropic optimal transport »
    Gonzalo Mena · Amin Nejatbakhsh · Erdem Varol · Jonathan Niles-Weed
  • 2021 : Sinkhorn EM: An Expectation-Maximizationalgorithm based on entropic optimal transport »
    Gonzalo Mena · Amin Nejatbakhsh · Erdem Varol · Jonathan Niles-Weed
  • 2022 Poster: The computational and learning benefits of Daleian neural networks »
    Adam Haber · Elad Schneidman
  • 2021 Poster: Deep Networks Provably Classify Data on Curves »
    Tingran Wang · Sam Buchanan · Dar Gilboa · John Wright
  • 2020 : Deep Networks and the Multiple Manifold Problem »
    Samuel Buchanan · Dar Gilboa · John Wright
  • 2019 : Poster Session »
    Pravish Sainath · Mohamed Akrout · Charles Delahunt · Nathan Kutz · Guangyu Robert Yang · Joseph Marino · L F Abbott · Nicolas Vecoven · Damien Ernst · andrew warrington · Michael Kagan · Kyunghyun Cho · Kameron Harris · Leopold Grinberg · John J. Hopfield · Dmitry Krotov · Taliah Muhammad · Erick Cobos · Edgar Walker · Jacob Reimer · Andreas Tolias · Alexander Ecker · Janaki Sheth · Yu Zhang · Maciej Wołczyk · Jacek Tabor · Szymon Maszke · Roman Pogodin · Dane Corneil · Wulfram Gerstner · Baihan Lin · Guillermo Cecchi · Jenna M Reinen · Irina Rish · Guillaume Bellec · Darjan Salaj · Anand Subramoney · Wolfgang Maass · Yueqi Wang · Ari Pakman · Jin Hyung Lee · Liam Paninski · Bryan Tripp · Colin Graber · Alex Schwing · Luke Prince · Gabriel Ocker · Michael Buice · Benjamin Lansdell · Konrad Kording · Jack Lindsey · Terrence Sejnowski · Matthew Farrell · Eric Shea-Brown · Nicolas Farrugia · Victor Nepveu · Jiwoong Im · Kristin Branson · Brian Hu · Ramakrishnan Iyer · Stefan Mihalas · Sneha Aenugu · Hananel Hazan · Sihui Dai · Tan Nguyen · Doris Tsao · Richard Baraniuk · Anima Anandkumar · Hidenori Tanaka · Aran Nayebi · Stephen Baccus · Surya Ganguli · Dean Pospisil · Eilif Muller · Jeffrey S Cheng · Gaël Varoquaux · Kamalaker Dadi · Dimitrios C Gklezakos · Rajesh PN Rao · Anand Louis · Christos Papadimitriou · Santosh Vempala · Naganand Yadati · Daniel Zdeblick · Daniela M Witten · Nicholas Roberts · Vinay Prabhu · Pierre Bellec · Poornima Ramesh · Jakob H Macke · Santiago Cadena · Guillaume Bellec · Franz Scherr · Owen Marschall · Robert Kim · Hannes Rapp · Marcio Fonseca · Oliver Armitage · Jiwoong Im · Thomas Hardcastle · Abhishek Sharma · Wyeth Bair · Adrian Valente · Shane Shang · Merav Stern · Rutuja Patil · Peter Wang · Sruthi Gorantla · Peter Stratton · Tristan Edwards · Jialin Lu · Martin Ester · Yurii Vlasov · Siavash Golkar
  • 2019 Poster: A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off »
    Yaniv Blumenfeld · Dar Gilboa · Daniel Soudry
  • 2017 : Poster Session »
    Shunsuke Horii · Heejin Jeong · Tobias Schwedes · Qing He · Ben Calderhead · Ertunc Erdil · Jaan Altosaar · Patrick Muchmore · Rajiv Khanna · Ian Gemp · Pengfei Zhang · Yuan Zhou · Chris Cremer · Maria DeYoreo · Alexander Terenin · Brendan McVeigh · Rachit Singh · Yaodong Yang · Erik Bodin · Trefor Evans · Henry Chai · Shandian Zhe · Jeffrey Ling · Vincent ADAM · Lars Maaløe · Andrew Miller · Ari Pakman · Josip Djolonga · Hong Ge
  • 2017 : Poster Spotlights »
    Francesco Locatello · Ari Pakman · Da Tang · Thomas Rainforth · Zalan Borsos · Marko Järvenpää · Eric Nalisnick · Gabriele Abbati · XIAOYU LU · Jonathan Huggins · Rachit Singh · Rui Luo
  • 2013 Poster: Bayesian Inference and Online Experimental Design for Mapping Neural Microcircuits »
    Ben Shababo · Brooks Paige · Ari Pakman · Liam Paninski
  • 2013 Spotlight: Bayesian Inference and Online Experimental Design for Mapping Neural Microcircuits »
    Ben Shababo · Brooks Paige · Ari Pakman · Liam Paninski
  • 2013 Poster: Auxiliary-variable Exact Hamiltonian Monte Carlo Samplers for Binary Distributions »
    Ari Pakman · Liam Paninski