Timezone: »

Kernel Memory Networks: A Unifying Framework for Memory Modeling
Georgios Iatropoulos · Johanni Brea · Wulfram Gerstner

Tue Nov 29 02:00 PM -- 04:00 PM (PST) @ Hall J #515

We consider the problem of training a neural network to store a set of patterns with maximal noise robustness. A solution, in terms of optimal weights and state update rules, is derived by training each individual neuron to perform either kernel classification or interpolation with a minimum weight norm. By applying this method to feed-forward and recurrent networks, we derive optimal models, termed kernel memory networks, that include, as special cases, many of the hetero- and auto-associative memory models that have been proposed over the past years, such as modern Hopfield networks and Kanerva's sparse distributed memory. We modify Kanerva's model and demonstrate a simple way to design a kernel memory network that can store an exponential number of continuous-valued patterns with a finite basin of attraction. The framework of kernel memory networks offers a simple and intuitive way to understand the storage capacity of previous memory models, and allows for new biological interpretations in terms of dendritic non-linearities and synaptic cross-talk.

Author Information

Georgios Iatropoulos (EPFL)
Johanni Brea (Swiss Federal Institute of Technology Lausanne)
Wulfram Gerstner (EPFL)

More from the Same Authors