ASIF-Net : Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection

Chongyi LI, Runmin CONG, Sam KWONG, Junhui HOU, Huazhu FU, Guopu ZHU, Dingwen ZHANG, Qingming HUANG

Research output: Journal PublicationsJournal Article (refereed)peer-review

184 Citations (Scopus)

Abstract

Salient object detection from RGB-D images is an important yet challenging vision task, which aims at detecting the most distinctive objects in a scene by combining color information and depth constraints. Unlike prior fusion manners, we propose an attention steered interweave fusion network (ASIF-Net) to detect salient objects, which progressively integrates cross-modal and cross-level complementarity from the RGB image and corresponding depth map via steering of an attention mechanism. Specifically, the complementary features from RGB-D images are jointly extracted and hierarchically fused in a dense and interweaved manner. Such a manner breaks down the barriers of inconsistency existing in the cross-modal data and also sufficiently captures the complementarity. Meanwhile, an attention mechanism is introduced to locate the potential salient regions in an attention-weighted fashion, which advances in highlighting the salient objects and suppressing the cluttered background regions. Instead of focusing only on pixelwise saliency, we also ensure that the detected salient objects have the objectness characteristics (e.g., complete structure and sharp boundary) by incorporating the adversarial learning that provides a global semantic constraint for RGB-D salient object detection. Quantitative and qualitative experiments demonstrate that the proposed method performs favorably against 17 state-of-the-art saliency detectors on four publicly available RGB-D salient object detection datasets. The code and results of our method are available at https://github.com/Li-Chongyi/ASIF-Net.
Original languageEnglish
Pages (from-to)88-100
JournalIEEE Transactions on Cybernetics
Volume51
Issue number1
Early online date13 Feb 2020
DOIs
Publication statusPublished - Jan 2021
Externally publishedYes

Bibliographical note

This work was supported in part by the Dr. Cong’s Project of the Fundamental Research Funds for the Central Universities under Grant 2019RC039, in part by the National Natural Science Foundation of China under Grant 61771334, Grant 61871342, Grant 61872350, Grant 61672443, Grant 61931008, Grant 61836002, and Grant U1636214, in part by the Hong Kong Research Grants Council General Research Funds under Grant 9042038 (CityU 11205314) and Grant 9042322 (CityU 11200116), in part by the Hong Kong Research Grants Council Early Career Schemes under Grant 9048123 (CityU 21211518), and in part by the China Postdoctoral Support Scheme for Innovative Talents under Grant BX20180236.

Keywords

  • Adversarial learning
  • depth cue
  • interweave fusion
  • residual attention
  • RGB-D images
  • saliency detection

Fingerprint

Dive into the research topics of 'ASIF-Net : Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection'. Together they form a unique fingerprint.

Cite this