Invited Talk: Computing and learning in the presence of neural noise

One key distinction between artificial and biological neural networks is the presence of noise, both intrinsic, e.g. due to synaptic failures, and extrinsic, arising through complex recurrent dynamics. Traditionally, this noise has been viewed as a ‘bug’, and the main computational challenge that the brain needs to face. More recently, it has been argued that circuit stochasticity may be a ‘feature', in that can be recruited for useful computations, such as representing uncertainty about the state of the world. Here we lay out a new argument for the role of stochasticity during learning. In particular, we use a mathematically tractable stochastic neural network model that allows us to derive local plasticity rules for optimizing a given global objective. This rule leads to representations that reflect both task structure and stimuli priors in interesting ways. Moreover, in this framework stochasticity is both a feature, as learning cannot happen in the absence of noise, and a bug, as the noise corrupts neural representations. Importantly, the network learns to use recurrent interactions to compensate for its negative effects, and maintain robust circuit function.