Skip to yearly menu bar Skip to main content

Workshop: Machine Learning with New Compute Paradigms

Adjoint Method: The Connection between Analog-based Equilibrium Propagation Architectures and Neural ODEs

Mohamed Watfa · Alberto Garcia-Ortiz

[ ] [ Project Page ]
Sat 16 Dec 9:25 a.m. PST — 10:30 a.m. PST


Analog neural networks (ANNs) hold significant potential for substantialreductions in power consumption in modern neural networks, particularly whenemploying the increasingly popular Energy-Based Models (EBMs) in tandem withthe local Equilibrium Propagation (EP) training algorithm. This paper analyzesthe relationship between this family of ANNs and the concept of Neural OrdinaryDifferential Equations (Neural ODEs). Using the adjoint method, we formallydemonstrate that ANN-EP can be derived from Neural ODEs by constraining thedifferential equations to those with a steady-state response. This findingopens avenues for the ANN-EP community to extend ANNs to non-steady-statescenarios. Additionally, it provides an efficient setting for NN-ODEs thatsignificantly reduces the training cost.

Chat is not available.