Timezone: »
Machine learning on tiny IoT devices based on microcontroller units (MCU) is appealing but challenging: the memory of microcontrollers is 2-3 orders of magnitude smaller even than mobile phones. We propose MCUNet, a framework that jointly designs the efficient neural architecture (TinyNAS) and the lightweight inference engine (TinyEngine), enabling ImageNet-scale inference on microcontrollers. TinyNAS adopts a two-stage neural architecture search approach that first optimizes the search space to fit the resource constraints, then specializes the network architecture in the optimized search space. TinyNAS can automatically handle diverse constraints (i.e. device, latency, energy, memory) under low search costs. TinyNAS is co-designed with TinyEngine, a memory-efficient inference library to expand the search space and fit a larger model. TinyEngine adapts the memory scheduling according to the overall network topology rather than layer-wise optimization, reducing the memory usage by 3.4×, and accelerating the inference by 1.7-3.3× compared to TF-Lite Micro [3] and CMSIS-NN [28]. MCUNet is the first to achieves >70% ImageNet top1 accuracy on an off-the-shelf commercial microcontroller, using 3.5× less SRAM and 5.7× less Flash compared to quantized MobileNetV2 and ResNet-18. On visual&audio wake words tasks, MCUNet achieves state-of-the-art accuracy and runs 2.4-3.4× faster than Mo- bileNetV2 and ProxylessNAS-based solutions with 3.7-4.1× smaller peak SRAM. Our study suggests that the era of always-on tiny machine learning on IoT devices has arrived.
Author Information
Ji Lin (MIT)
Wei-Ming Chen (National Taiwan University)
Yujun Lin (MIT)
john cohn (MIT-IBM Watson AI Lab, IBM Research)
John Cohn is an IBM Fellow in the MIT-IBM Watson AI Research Group based in Cambridge, MA. John earned a BSEE MIT, and a Ph.D in Computer Engineering from Carnegie Mellon University He has authored more than 30 technical papers, contributed to four books and has more than 100 worldwide patents. In 2005 John was elected a Fellow of the IEEE. John is active in education issues at a local, state and national level. He is so passionate about promoting STEM careers that he spent 59 days living and inventing in an abandoned steel mill as part of Discovery Channel’s technical survival show “The Colony”. John lives with his family in a restored 19th century schoolhouse in Jonesville Vermont and is eager to share his love of science and technology with anyone who will listen.
Chuang Gan (MIT-IBM Watson AI Lab)
Song Han (MIT)
Related Events (a corresponding poster, oral, or spotlight)
-
2020 Poster: MCUNet: Tiny Deep Learning on IoT Devices »
Fri. Dec 11th 05:00 -- 07:00 AM Room Poster Session 6 #1786
More from the Same Authors
-
2021 : ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation »
Chuang Gan · Jeremy Schwartz · Seth Alter · Damian Mrowca · Martin Schrimpf · James Traer · Julian De Freitas · Jonas Kubilius · Abhishek Bhandwaldar · Nick Haber · Megumi Sano · Kuno Kim · Elias Wang · Michael Lingelbach · Aidan Curtis · Kevin Feigelis · Daniel Bear · Dan Gutfreund · David Cox · Antonio Torralba · James J DiCarlo · Josh Tenenbaum · Josh McDermott · Dan Yamins -
2021 : STAR: A Benchmark for Situated Reasoning in Real-World Videos »
Bo Wu · Shoubin Yu · Zhenfang Chen · Josh Tenenbaum · Chuang Gan -
2021 Poster: Memory-efficient Patch-based Inference for Tiny Deep Learning »
Ji Lin · Wei-Ming Chen · Han Cai · Chuang Gan · Song Han -
2021 Poster: Dynamic Visual Reasoning by Learning Differentiable Physics Models from Video and Language »
Mingyu Ding · Zhenfang Chen · Tao Du · Ping Luo · Josh Tenenbaum · Chuang Gan -
2021 Poster: PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning »
Yining Hong · Li Yi · Josh Tenenbaum · Antonio Torralba · Chuang Gan -
2021 Poster: When does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning? »
Lijie Fan · Sijia Liu · Pin-Yu Chen · Gaoyuan Zhang · Chuang Gan -
2021 Poster: Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning »
Ligeng Zhu · Hongzhou Lin · Yao Lu · Yujun Lin · Song Han -
2021 : ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation »
Chuang Gan · Jeremy Schwartz · Seth Alter · Damian Mrowca · Martin Schrimpf · James Traer · Julian De Freitas · Jonas Kubilius · Abhishek Bhandwaldar · Nick Haber · Megumi Sano · Kuno Kim · Elias Wang · Michael Lingelbach · Aidan Curtis · Kevin Feigelis · Daniel Bear · Dan Gutfreund · David Cox · Antonio Torralba · James J DiCarlo · Josh Tenenbaum · Josh McDermott · Dan Yamins -
2020 Poster: Differentiable Augmentation for Data-Efficient GAN Training »
Shengyu Zhao · Zhijian Liu · Ji Lin · Jun-Yan Zhu · Song Han -
2020 Poster: TinyTL: Reduce Memory, Not Parameters for Efficient On-Device Learning »
Han Cai · Chuang Gan · Ligeng Zhu · Song Han -
2020 : Neurosymbolic Visual Reasoning »
Chuang Gan -
2019 : Poster Session 2 »
Hanson Wang · Yujun Lin · Yixiao Duan · Aditya Paliwal · Ameer Haj-Ali · Ryan Marcus · Tom Hope · Qiumin Xu · Nham Le · Yuxiang Sun · Ross Cutler · Vikram Nathan · Min Sun -
2019 : Contributed Talk 4: Neural Hardware Architecture Search »
Yujun Lin -
2019 : Hardware-aware Neural Architecture Design for Small and Fast Models: from 2D to 3D »
Song Han -
2019 : Posters and Coffee »
Sameer Kumar · Tomasz Kornuta · Oleg Bakhteev · Hui Guan · Xiaomeng Dong · Minsik Cho · Soeren Laue · Theodoros Vasiloudis · Andreea Anghel · Erik Wijmans · Zeyuan Shang · Oleksii Kuchaiev · Ji Lin · Susan Zhang · Ligeng Zhu · Beidi Chen · Vinu Joseph · Jialin Ding · Jonathan Raiman · Ahnjae Shin · Vithursan Thangarasa · Anush Sankaran · Akhil Mathur · Martino Dazzi · Markus Löning · Darryl Ho · Emanuel Zgraggen · Supun Nakandala · Tomasz Kornuta · Rita Kuznetsova -
2019 Poster: Park: An Open Platform for Learning-Augmented Computer Systems »
Hongzi Mao · Parimarjan Negi · Akshay Narayan · Hanrui Wang · Jiacheng Yang · Haonan Wang · Ryan Marcus · Ravichandra Addanki · Mehrdad Khani Shirkoohi · Songtao He · Vikram Nathan · Frank Cangialosi · Shaileshh Venkatakrishnan · Wei-Hung Weng · Song Han · Tim Kraska · Dr.Mohammad Alizadeh -
2019 Poster: Deep Leakage from Gradients »
Ligeng Zhu · Zhijian Liu · Song Han -
2019 Poster: Cross-channel Communication Networks »
Jianwei Yang · Zhile Ren · Chuang Gan · Hongyuan Zhu · Devi Parikh -
2019 Poster: Visual Concept-Metaconcept Learning »
Chi Han · Jiayuan Mao · Chuang Gan · Josh Tenenbaum · Jiajun Wu -
2019 Poster: Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement »
Chao Yang · Xiaojian Ma · Wenbing Huang · Fuchun Sun · Huaping Liu · Junzhou Huang · Chuang Gan -
2019 Spotlight: Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement »
Chao Yang · Xiaojian Ma · Wenbing Huang · Fuchun Sun · Huaping Liu · Junzhou Huang · Chuang Gan -
2019 Poster: Point-Voxel CNN for Efficient 3D Deep Learning »
Zhijian Liu · Haotian Tang · Yujun Lin · Song Han -
2019 Spotlight: Point-Voxel CNN for Efficient 3D Deep Learning »
Zhijian Liu · Haotian Tang · Yujun Lin · Song Han -
2018 : Panel disucssion »
Max Welling · Tim Genewein · Edwin Park · Song Han -
2018 : Prof. Song Han »
Song Han -
2018 : Poster spotlight session. »
Abdullah Salama · Wei-Cheng Chang · Aidan Gomez · Raphael Tang · FUXUN YU · Zhendong Zhang · Yuxin Zhang · Ji Lin · Stephen Tiedemann · Kun Bai · Sivaramakrishnan Sankarapandian · Marton Havasi · Jack Turner · Hsin-Pai Cheng · Yue Wang · Xiaofan Xu · Ruizhou Ding · Haoji Hu · Mohammad Shafiee · Christopher Blake · Chieh-Chi Kao · Daniel Kang · Yew Ken Chia · Amir Ashouri · Sourya Basu · Simon Wiedemann · Thorsten Laude -
2018 : Bandwidth efficient deep learning by model compression »
Song Han -
2018 Poster: Weakly Supervised Dense Event Captioning in Videos »
Xin Wang · Wenbing Huang · Chuang Gan · Jingdong Wang · Wenwu Zhu · Junzhou Huang -
2018 Poster: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2018 Spotlight: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2017 Poster: Runtime Neural Pruning »
Ji Lin · Yongming Rao · Jiwen Lu · Jie Zhou