RGB-D Salient Object Detection with Cross-Modality Modulation and Selection

Chongyi LI, Runmin CONG*, Yongri PIAO, Qianqian XU, Chen Change LOY

*Corresponding author for this work

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Referred Conference Paperpeer-review

109 Citations (Scopus)

Abstract

We present an effective method to progressively integrate and refine the cross-modality complementarities for RGB-D salient object detection (SOD). The proposed network mainly solves two challenging issues: 1) how to effectively integrate the complementary information from RGB image and its corresponding depth map, and 2) how to adaptively select more saliency-related features. First, we propose a cross-modality feature modulation (cmFM) module to enhance feature representations by taking the depth features as prior, which models the complementary relations of RGB-D data. Second, we propose an adaptive feature selection (AFS) module to select saliency-related features and suppress the inferior ones. The AFS module exploits multi-modality spatial feature fusion with the self-modality and cross-modality interdependencies of channel features are considered. Third, we employ a saliency-guided position-edge attention (sg-PEA) module to encourage our network to focus more on saliency-related regions. The above modules as a whole, called cmMS block, facilitates the refinement of saliency features in a coarse-to-fine fashion. Coupled with a bottom-up inference, the refined saliency features enable accurate and edge-preserving SOD. Extensive experiments demonstrate that our network outperforms state-of-the-art saliency detectors on six popular RGB-D SOD benchmarks.

Original languageEnglish
Title of host publicationComputer Vision – ECCV 2020 : 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII
EditorsAndrea VEDALDI, Horst BISCHOF, Thomas BROX, Jan-Michael FRAHM
PublisherSpringer Science and Business Media Deutschland GmbH
Pages225-241
Number of pages17
ISBN (Electronic)9783030585983
ISBN (Print)9783030585976
DOIs
Publication statusPublished - 2020
Externally publishedYes
Event16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom
Duration: 23 Aug 202028 Aug 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer
Volume12353
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349
NameECCV: European Conference on Computer Vision
PublisherSpringer

Conference

Conference16th European Conference on Computer Vision, ECCV 2020
Country/TerritoryUnited Kingdom
CityGlasgow
Period23/08/2028/08/20

Bibliographical note

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.

Fingerprint

Dive into the research topics of 'RGB-D Salient Object Detection with Cross-Modality Modulation and Selection'. Together they form a unique fingerprint.

Cite this