Task-aware swapping for efficient DNN inference on DRAM-constrained edge systems

Cheng JI, Zongwei ZHU*, Xianmin WANG*, Wenjie ZHAI, Xuemei ZONG, Anqi CHEN, Mingliang ZHOU

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

2 Citations (Scopus)

Abstract

Object detection at the edge side is a common task in various environments. The deployment of convolutional neural networks in intelligent edge systems is very challenging because of the highly constrained main-memory space. This study aims at operating neural networks with a reduced memory requirement. The basic idea is that tasks of the same type would involve the same critical subnetwork. We propose identifying the critical network connections by considering the importance of channels. During runtime, the proposed method detects the task types and timely swaps the model parameters of the critical subnetworks from the external storage into dynamic random access memory (DRAM). Compared with conventional network pruning, the proposed approach further reduced the DRAM requirement by 34.6% while maintaining a high inference accuracy.

Original languageEnglish
Pages (from-to)8155-8169
Number of pages15
JournalInternational Journal of Intelligent Systems
Volume37
Issue number10
Early online date1 Jun 2022
DOIs
Publication statusPublished - Oct 2022
Externally publishedYes

Bibliographical note

ACKNOWLEDGMENTS:
This study was partially supported by the National Natural Science Foundation of China (Nos. 62102179, 62072127, 62002076, 62176027, and 62102179), Natural Science Foundation of Jiangsu Province (No. BK20200462), Project No. 6142111180404 supported by CNKLSTISS, Science and Technology Program of Guangzhou, China (No. 202002030131), Guangdong Basic and Applied Basic Research Fund Joint Fund Youth Fund (No. 2019A1515110213), Open Fund Project of Fujian Provincial Key Laboratory of Information Processing, Intelligent Control (Minjiang University; No. MJUKF-IPIC202101), and General Program of National Natural Science Foundation of Chongqing under Grant No. cstc2020jcyj-msxmX0790. We also thank Huanghe Liu and Fan Wu for their help for this work.

Keywords

  • deep neural network
  • edge intelligent systems
  • inference
  • object detection
  • resource constraint

Fingerprint

Dive into the research topics of 'Task-aware swapping for efficient DNN inference on DRAM-constrained edge systems'. Together they form a unique fingerprint.

Cite this