Compacting Deep Neural Networks for Internet of Things: Methods and Applications

Ke ZHANG*, Hanbo YING, Hong Ning DAI, Lin LI, Yuanyuan PENG, Keyi GUO, Hongfang YU

*Corresponding author for this work

Research output: Journal PublicationsReview articlepeer-review

Abstract

Deep neural networks (DNNs) have shown great success in completing complex tasks. However, DNNs inevitably bring high computational cost and storage consumption due to the complexity of hierarchical structures, thereby hindering their wide deployment in Internet-of-Things (IoT) devices, which have limited computational capability and storage capacity. Therefore, it is a necessity to investigate the technologies to compact DNNs. Despite tremendous advances in compacting DNNs, few surveys summarize compacting-DNNs technologies, especially for IoT applications. Hence, this article presents a comprehensive study on compacting-DNNs technologies. We categorize compacting-DNNs technologies into three major types: 1) network model compression; 2) knowledge distillation (KD); and 3) modification of network structures. We also elaborate on the diversity of these approaches and make side-by-side comparisons. Moreover, we discuss the applications of compacted DNNs in various IoT applications and outline future directions.

Original languageEnglish
Article number9367271
Pages (from-to)11935-11959
Number of pages25
JournalIEEE Internet of Things Journal
Volume8
Issue number15
DOIs
Publication statusPublished - 1 Aug 2021
Externally publishedYes

Bibliographical note

Funding Information:
Manuscript received September 29, 2020; revised February 6, 2021; accepted February 28, 2021. Date of publication March 3, 2021; date of current version July 23, 2021. This work was supported in part by the Sichuan Science and Technology Program under Grant 2019YFG0405; in part by the Project of Science and Technology on Electronic Information Control Laboratory; in part by the Joint Key Research and Development Project between Sichuan and Chongqing under Grant cstc2020jscx-cylhX0004; and in part by the Macao Science and Technology Development Fund under Macao Funding Scheme for Key Research and Development Projects under Grant 0025/2019/AKP. (Corresponding author: Ke Zhang.) Ke Zhang and Yuanyuan Peng are with the School of Computer Science and Engineering, and Science and Technology on Electronic Information Control Laboratory, and also with the School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China (e-mail: kezhang@uestc.edu.cn; pengyuanyuan@std.uestc.edu.cn).

Publisher Copyright:
© 2014 IEEE.

Keywords

  • Deep learning (DL)
  • deep neural networks (DNNs)
  • Internet of Things (IoT)
  • model compression

Fingerprint

Dive into the research topics of 'Compacting Deep Neural Networks for Internet of Things: Methods and Applications'. Together they form a unique fingerprint.

Cite this