Timezone: »
Poster
Holomorphic Equilibrium Propagation Computes Exact Gradients Through Finite Size Oscillations
Axel Laborieux · Friedemann Zenke
Equilibrium propagation (EP) is an alternative to backpropagation (BP) that allows the training of deep neural networks with local learning rules. It thus provides a compelling framework for training neuromorphic systems and understanding learning in neurobiology. However, EP requires infinitesimal teaching signals, thereby limiting its applicability to noisy physical systems. Moreover, the algorithm requires separate temporal phases and has not been applied to large-scale problems. Here we address these issues by extending EP to holomorphic networks. We show analytically that this extension naturally leads to exact gradients for finite-amplitude teaching signals. Importantly, the gradient can be computed as the first Fourier coefficient from finite neuronal activity oscillations in continuous time without requiring separate phases. Further, we demonstrate in numerical simulations that our approach permits robust estimation of gradients in the presence of noise and that deeper models benefit from the finite teaching signals. Finally, we establish the first benchmark for EP on the ImageNet $32 \times 32$ dataset and show that it matches the performance of an equivalent network trained with BP. Our work provides analytical insights that enable scaling EP to large-scale problems and establishes a formal framework for how oscillations could support learning in biological and neuromorphic systems.
Author Information
Axel Laborieux (Friedrich Miescher Institute for Biomedical Research)

I am a Postdoctoral researcher at the Friedrich Miescher Institute in Computational Neuroscience in the team of Friedemann Zenke. Previously a PhD candidate in hardware-oriented AI at the Center for Nanoscience and Nanotechnology, attached to Paris-Saclay University.
Friedemann Zenke (Friedrich Miescher Institute)
More from the Same Authors
-
2023 Poster: Isotropic loss design for non-contrastive SSL »
Manu Srinath Halvagal · Axel Laborieux · Friedemann Zenke -
2023 Poster: Credit assignment through disinhibitory control of Hebbian plasticity »
Julian Rossbroich · Friedemann Zenke -
2022 Panel: Panel 1C-1: Learning Neural Set… & Holomorphic Equilibrium Propagation… »
Axel Laborieux · Yatao Bian -
2020 Poster: A meta-learning approach to (re)discover plasticity rules that carve a desired function into a neural network »
Basile Confavreux · Friedemann Zenke · Everton Agnes · Timothy Lillicrap · Tim Vogels -
2020 Spotlight: A meta-learning approach to (re)discover plasticity rules that carve a desired function into a neural network »
Basile Confavreux · Friedemann Zenke · Everton Agnes · Timothy Lillicrap · Tim Vogels