Noise-Tolerant Hardware-Aware Pruning for Deep Neural Networks

Existing hardware-aware pruning methods for deep neural networks do not take the uncertain execution environment of low-end hardware into consideration. That makes those methods unreliable, since the hardware environments they used for evaluating the pruned models contain uncertainty and thus the performance values contain noise. To deal with this problem, this paper proposes noise-tolerant hardware-aware pruning, i.e., NT-HP. It uses a population-based idea to iteratively generate pruned models. Each pruned model is sent to realistic low-end hardware for performance evaluations. For the noisy values of performance indicators collected from hardware, a threshold for comparison is set, where only the pruned models with significantly better performances are kept in the next generation. Our experimental results show that with the noise-tolerant technique involved, NT-HP can get better pruned models in the uncertain execution environment of low-end hardware. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.