Timezone: »

 
Poster
E2-Train: Training State-of-the-art CNNs with Over 80% Less Energy
Ziyu Jiang · Yue Wang · Xiaohan Chen · Pengfei Xu · Yang Zhao · Yingyan Lin · Zhangyang Wang

Thu Dec 12 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #158
in Deep Learning -- Efficient Training Methods »

Convolutional neural networks (CNNs) have been increasingly deployed to edge devices. Hence, many efforts have been made towards efficient CNN inference on resource-constrained platforms. This paper attempts to explore an orthogonal direction: how to conduct more energy-efficient training of CNNs, so as to enable on-device training? We strive to reduce the energy cost during training, by dropping unnecessary computations, from three complementary levels: stochastic mini-batch dropping on the data level; selective layer update on the model level; and sign prediction for low-cost, low-precision back-propagation, on the algorithm level. Extensive simulations and ablation studies, with real energy measurements from an FPGA board, confirm the superiority of our proposed strategies and demonstrate remarkable energy savings for training. For example, when training ResNet-74 on CIFAR-10, we achieve aggressive energy savings of >90% and >60%, while incurring a top-1 accuracy loss of only about 2% and 1.2%, respectively. When training ResNet-110 on CIFAR-100, an over 84% training energy saving is achieved without degrading inference accuracy.

Keywords: Efficient Training Methods · Deep Learning
[ Paper]  [ Poster] 

Author Information

Ziyu Jiang (Texas A&M University)
Yue Wang (Rice University)
Xiaohan Chen (Texas A&M University)
Pengfei Xu (Rice University)
Yang Zhao (Rice University)
Yingyan Lin (Rice University)

The assistant professor working on energy-efficient machine learning systems

Zhangyang Wang (TAMU)

More from the Same Authors