Timezone: »
Most large-scale network models use neurons with static nonlinearities that produce analog output, despite the fact that information processing in the brain is predominantly carried out by dynamic neurons that produce discrete pulses called spikes. Research in spike-based computation has been impeded by the lack of efficient supervised learning algorithm for spiking neural networks. Here, we present a gradient descent method for optimizing spiking network models by introducing a differentiable formulation of spiking dynamics and deriving the exact gradient calculation. For demonstration, we trained recurrent spiking networks on two dynamic tasks: one that requires optimizing fast (~ millisecond) spike-based interactions for efficient encoding of information, and a delayed-memory task over extended duration (~ second). The results show that the gradient descent approach indeed optimizes networks dynamics on the time scale of individual spikes as well as on behavioral time scales. In conclusion, our method yields a general purpose supervised learning algorithm for spiking neural networks, which can facilitate further investigations on spike-based computations.
Author Information
Ben Huh (MIT-IBM AI Center)
Terrence Sejnowski (Salk Institute)
More from the Same Authors
-
2013 Session: Oral Session 3 »
Terrence Sejnowski -
2012 Invited Talk (Posner Lecture): Suspicious Coincidences in the Brain »
Terrence Sejnowski -
2011 Session: Opening Remarks and Awards »
Terrence Sejnowski · Peter Bartlett · Fernando Pereira -
2010 Placeholder: Opening Remarks »
Terrence Sejnowski · Neil D Lawrence -
2010 Talk: Opening Remarks and Awards »
Richard Zemel · Terrence Sejnowski · John Shawe-Taylor -
2009 Workshop: The Curse of Dimensionality Problem: How Can the Brain Solve It? »
Simon Haykin · Terrence Sejnowski · Steven W Zucker -
2008 Workshop: Cortical Microcircuits and their Computational Functions »
Tomaso Poggio · Terrence Sejnowski -
2006 Workshop: Decoding the neural code »
Eric Thomson · Bill Kristan · Terrence Sejnowski