Timezone: »

 
Poster
ANODEV2: A Coupled Neural ODE Framework
Tianjun Zhang · Zhewei Yao · Amir Gholami · Joseph Gonzalez · Kurt Keutzer · Michael Mahoney · George Biros

Thu Dec 12 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #131
in Deep Learning -- CNN Architectures »

It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, in which other discretization schemes and/or adaptive time stepping techniques can be used to improve the performance of residual networks. Here, we propose \OURS, which extends this approach by introducing a framework that allows ODE-based evolution for both the weights and the activations, in a coupled formulation. Such an approach provides more modeling flexibility, and it can help with generalization performance. We present the formulation of \OURS, derive optimality conditions, and implement the coupled framework in PyTorch. We present empirical results using several different configurations of \OURS, testing them on the CIFAR-10 dataset. We report results showing that our coupled ODE-based framework is indeed trainable, and that it achieves higher accuracy, compared to the baseline ResNet network and the recently-proposed Neural ODE approach.

Keywords: Deep Learning · Computer Vision · Applications
[ Paper]  [ Slides] 

Author Information

Tianjun Zhang (University of California, Berkeley)
Zhewei Yao (UC Berkeley)
Amir Gholami (University of California, Berkeley)
Joseph Gonzalez (UC Berkeley)
Kurt Keutzer (EECS, UC Berkeley)
Michael Mahoney (UC Berkeley)
George Biros (University of Texas at Austin)

More from the Same Authors