Training a Predictive Coding Network on ImageNet using Equilibrium Propagation

Equilibrium Propagation (EP) is a training framework for energy-based models (EBMs) that has attracted interest in the context of neuromorphic computing platforms such as continuous Hopfield networks, nonlinear resistive networks and coupled phase oscillators. However, EP's practical applications have so far remained limited to relatively small-scale problems. Predictive coding networks (PCNs), another class of EBMs rooted in computational neuroscience, are typically trained with a specialized algorithm and have likewise not yet been demonstrated at large scale. In this work, we develop a more effective training method for PCNs which combines the centered variant of EP with a novel equilibration scheme for PCNs. Using this approach, we train a 10-layer convolutional PCN (VGG10) on full-size ImageNet, achieving 13.23% test error rate on the top-5 classification task, close to the 12.2% backpropagation baseline. To our knowledge, this is the first demonstration of both PCNs and EP-based training at ImageNet scale. These results significantly extend the scalability of both approaches and suggest that the primary challenges in scaling EP in other EBMs may not be attributed to inherent limitations of the EP framework.