Point-aware Interaction and CNN-induced Refinement Network for RGB-D Salient Object Detection

Runmin CONG, Hongyu LIU*, Wei ZHANG, Feng ZHENG, Ran SONG, Sam KWONG, Chen ZHANG

*Corresponding author for this work

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Referred Conference Paperpeer-review

5 Citations (Scopus)

Abstract

By integrating complementary information from RGB image and depth map, the ability of salient object detection (SOD) for complex and challenging scenes can be improved. In recent years, the important role of Convolutional Neural Networks (CNNs) in feature extraction and cross-modality interaction has been fully explored, but it is still insufficient in modeling global long-range dependencies of self-modality and cross-modality. To this end, we introduce CNNs-assisted Transformer architecture and propose a novel RGB-D SOD network with Point-aware Interaction and CNN-induced Refinement (PICR-Net). On the one hand, considering the prior correlation between RGB modality and depth modality, an attention-triggered cross-modality point-aware interaction (CmPI) module is designed to explore the feature interaction of different modalities with positional constraints. On the other hand, in order to alleviate the block effect and detail destruction problems brought by the Transformer naturally, we design a CNN-induced refinement (CNNR) unit for content refinement and supplementation. Extensive experiments on five RGB-D SOD datasets show that the proposed network achieves competitive results in both quantitative and qualitative comparisons. Our code is publicly available at: https://github.com/rmcong/PICR-Net_ACMMM23.
Original languageEnglish
Title of host publicationMM 2023 - Proceedings of the 31st ACM International Conference on Multimedia
PublisherAssociation for Computing Machinery (ACM)
Pages406-416
Number of pages11
ISBN (Electronic)9798400701085
ISBN (Print)9798400701085
DOIs
Publication statusPublished - 27 Oct 2023
Externally publishedYes
EventThe 31st ACM International Conference on Multimedia - Ottawa, Canada
Duration: 29 Oct 20233 Nov 2023

Conference

ConferenceThe 31st ACM International Conference on Multimedia
Abbreviated titleMM '23
Country/TerritoryCanada
CityOttawa
Period29/10/233/11/23

Bibliographical note

Publisher Copyright:
© 2023 ACM.

Keywords

  • cnns-assisted transformer architecture
  • point-aware interaction
  • rgb-d images
  • salient object detection

Fingerprint

Dive into the research topics of 'Point-aware Interaction and CNN-induced Refinement Network for RGB-D Salient Object Detection'. Together they form a unique fingerprint.

Cite this