Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

Baian CHEN; Zhilei CHEN; Xiaowei HU; Jun XU; Haoran XIE; Jing QIN; Mingqiang WEI

doi:10.1145/3597612

Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

Baian CHEN, Zhilei CHEN, Xiaowei HU, Jun XU, Haoran XIE^*, Jing QIN, Mingqiang WEI^*

^*Corresponding author for this work

Lingnan University

Research output: Journal Publications › Journal Article (refereed) › peer-review

15 Citations (Scopus)

Abstract

Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this article, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over 17 methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at https://github.com/chenbaian-cs/DMPNet.

Original language	English
Article number	3597612
Number of pages	21
Journal	ACM Transactions on Multimedia Computing, Communications and Applications
Volume	20
Issue number	1
Early online date	19 May 2023
DOIs	https://doi.org/10.1145/3597612
Publication status	Published - Jan 2024

Bibliographical note

Publisher Copyright:
© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Funding

This work was supported by the Shenzhen Science and Technology Program (No. JCYJ20220818103401003, No. JCYJ20220530172403007), by the General Program of Natural Science Foundation of Guangdong Province (No. 2022A1515010170), by the Free Exploration of Basic Research Project, Local Science and Technology Development Fund Guided by the Central Government of China (No. 2021Szvup060), and the Research Grant entitled "Self-Supervised Learning for Medical Images"' (No. 871228) and Shenzhen University-Lingnan University Joint Research Programme (SZU-LU006/ 2122) of Lingnan University, Hong Kong.

Keywords

RGB-D salient object detection
dynamic message propagation
cross-modal learning
depth feature propagation

Access to Document

10.1145/3597612

https://dl.acm.org/doi/10.1145/3597612

Cite this

@article{ef23dd52d9624f9ea56268054d3de703,

title = "Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection",

abstract = "Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this article, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over 17 methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at https://github.com/chenbaian-cs/DMPNet.",

keywords = "RGB-D salient object detection, dynamic message propagation, cross-modal learning, depth feature propagation",

author = "Baian CHEN and Zhilei CHEN and Xiaowei HU and Jun XU and Haoran XIE and Jing QIN and Mingqiang WEI",

note = "Publisher Copyright: {\textcopyright} 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.",

year = "2024",

month = jan,

doi = "10.1145/3597612",

language = "English",

volume = "20",

journal = "ACM Transactions on Multimedia Computing, Communications and Applications",

issn = "1551-6857",

publisher = "Association for Computing Machinery (ACM)",

number = "1",

}

TY - JOUR

T1 - Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

AU - CHEN, Baian

AU - CHEN, Zhilei

AU - HU, Xiaowei

AU - XU, Jun

AU - XIE, Haoran

AU - QIN, Jing

AU - WEI, Mingqiang

PY - 2024/1

Y1 - 2024/1

N2 - Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this article, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over 17 methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at https://github.com/chenbaian-cs/DMPNet.

AB - Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this article, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over 17 methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at https://github.com/chenbaian-cs/DMPNet.

KW - RGB-D salient object detection

KW - dynamic message propagation

KW - cross-modal learning

KW - depth feature propagation

UR - http://www.scopus.com/inward/record.url?scp=85173225044&partnerID=8YFLogxK

U2 - 10.1145/3597612

DO - 10.1145/3597612

M3 - Journal Article (refereed)

SN - 1551-6857

VL - 20

JO - ACM Transactions on Multimedia Computing, Communications and Applications

JF - ACM Transactions on Multimedia Computing, Communications and Applications

IS - 1

M1 - 3597612

ER -

Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Developing Diagnosis System based on Self-Supervised Adversarial Learning for Babies with Congenital Heart Defects

Self-Supervised Learning for Medical Images

Cite this

Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Projects

Developing Diagnosis System based on Self-Supervised Adversarial Learning for Babies with Congenital Heart Defects

Self-Supervised Learning for Medical Images

Cite this