WaveNet : Wavelet Network With Knowledge Distillation for RGB-T Salient Object Detection

Wujie ZHOU, Fan SUN, Qiuping JIANG*, Runmin CONG*, Jenq-Neng HWANG

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

30 Citations (Scopus)

Abstract

In recent years, various neural network architectures for computer vision have been devised, such as the visual transformer and multilayer perceptron (MLP). A transformer based on an attention mechanism can outperform a traditional convolutional neural network. Compared with the convolutional neural network and transformer, the MLP introduces less inductive bias and achieves stronger generalization. In addition, a transformer shows an exponential increase in the inference, training, and debugging times. Considering a wave function representation, we propose the WaveNet architecture that adopts a novel vision task-oriented wavelet-based MLP for feature extraction to perform salient object detection in RGB (red-green-blue)-thermal infrared images. In addition, we apply knowledge distillation to a transformer as an advanced teacher network to acquire rich semantic and geometric information and guide WaveNet learning with this information. Following the shortest-path concept, we adopt the Kullback-Leibler distance as a regularization term for the RGB features to be as similar to the thermal infrared features as possible. The discrete wavelet transform allows for the examination of frequency-domain features in a local time domain and time-domain features in a local frequency domain. We apply this representation ability to perform cross-modality feature fusion. Specifically, we introduce a progressively cascaded sine-cosine module for cross-layer feature fusion and use low-level features to obtain clear boundaries of salient objects through the MLP. Results from extensive experiments indicate that the proposed WaveNet achieves impressive performance on benchmark RGB-thermal infrared datasets. The results and code are publicly available at https://github.com/nowander/WaveNet.

Original languageEnglish
Pages (from-to)3027-3039
Number of pages13
JournalIEEE Transactions on Image Processing
Volume32
Early online date16 May 2023
DOIs
Publication statusPublished - 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 1992-2012 IEEE.

Keywords

  • discrete wavelet transform
  • edge-aware module
  • knowledge distillation
  • progressively stretched sine-cosine module
  • Wavelet

Fingerprint

Dive into the research topics of 'WaveNet : Wavelet Network With Knowledge Distillation for RGB-T Salient Object Detection'. Together they form a unique fingerprint.

Cite this