Timezone: »

 
Poster
On the Stability and Scalability of Node Perturbation Learning
Naoki Hiratani · Yash Mehta · Timothy Lillicrap · Peter E Latham

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #302
in Poster Session 3 »

To survive, animals must adapt synaptic weights based on external stimuli and rewards. And they must do so using local, biologically plausible, learning rules -- a highly nontrivial constraint. One possible approach is to perturb neural activity (or use intrinsic, ongoing noise to perturb it), determine whether performance increases or decreases, and use that information to adjust the weights. This algorithm -- known as node perturbation -- has been shown to work on simple problems, but little is known about either its stability or its scalability with respect to network size. We investigate these issues both analytically, in deep linear networks, and numerically, in deep nonlinear ones.We show analytically that in deep linear networks with one hidden layer, both learning time and performance depend very weakly on hidden layer size. However, unlike stochastic gradient descent, when there is model mismatch between the student and teacher networks, node perturbation is always unstable. The instability is triggered by weight diffusion, which eventually leads to very large weights. This instability can be suppressed by weight normalization, at the cost of bias in the learning rule. We confirm numerically that a similar instability, and to a lesser extent scalability, exist in deep nonlinear networks trained on both a motor control task and image classification tasks. Our study highlights the limitations and potential of node perturbation as a biologically plausible learning rule in the brain.

Keywords: Biologically Plausible Deep Networks · Neuroscience · Dynamical Systems · Credit Assignment

Author Information

Naoki Hiratani (Harvard University)
Yash Mehta (HHMI Janelia Research Campus)

Hi! I’m currently a research engineer working on challenging neural architecture search research under the supervision of Prof **Frank Hutter** (ELLIS Fellow). Previously, I was a researcher at the *Gatsby Computational Neuroscience Unit* at UCL, where I was working on evaluating biologically plausible perturbation-based learning algorithms to train deep networks under the guidance of **Prof Peter Latham** (Gatsby) and **Tim Lillicrap** (DeepMind). In the past, I’ve also worked on deep learning-based personality detection from text with **Prof Erik Cambria** (NTU Singapore). I thoroughly enjoy coding and working on hard algorithmic problems.

Timothy Lillicrap (DeepMind & UCL)
Peter E Latham (Gatsby Unit, UCL)

More from the Same Authors