Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

Baian CHEN; Zhilei CHEN; Xiaowei HU; Jun XU; Haoran XIE; Jing QIN; Mingqiang WEI

doi:10.1145/3597612

Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

Baian CHEN, Zhilei CHEN, Xiaowei HU, Jun XU, Haoran XIE^*, Jing QIN, Mingqiang WEI^*

^*Corresponding author for this work

Lingnan University

Research output: Journal Publications › Journal Article (refereed) › peer-review

Abstract

Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this paper, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately, and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over seventeen methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at: https://github.com/chenbaian-cs/DMPNet.

Original language	English
Number of pages	21
Journal	ACM Transactions on Multimedia Computing, Communications and Applications
DOIs	https://doi.org/10.1145/3597612
Publication status	E-pub ahead of print - 19 May 2023

Bibliographical note

This work was supported by the Shenzhen Science and Technology Program (No.JCYJ20220818103401003, No. JCYJ20220530172403007), by the General Program of Natural Science Foundation of Guangdong Province (No. 2022A1515010170), by the Free Exploration of Basic Research Project, Local Science and Technology Development Fund Guided by the Central Government of China (No.2021Szvup060), and the Research Grant entitled "Self Supervised Learning for Medical Images" (No.871228) and Shenzhen University-Lingnan University Joint Research Programme (SZU-LU006/2122) of Lingnan University, Hong Kong.

Keywords

RGB-D salient object detection
dynamic message propagation
cross-modal learning
depth feature propagation

Access to Document

10.1145/3597612

https://dl.acm.org/doi/10.1145/3597612

Developing Diagnosis System based on Self-Supervised Adversarial Learning for Babies with Congenital Heart Defects
XIE, H.
LU Research Committee
1/07/22 → 30/06/23
Project: Grant Research
Self-Supervised Learning for Medical Images
XIE, H.
1/09/21 → 31/08/24
Project: Grant Research

Cite this

@article{ef23dd52d9624f9ea56268054d3de703,

title = "Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection",

abstract = "Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this paper, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately, and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over seventeen methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at: https://github.com/chenbaian-cs/DMPNet.",

keywords = "RGB-D salient object detection, dynamic message propagation, cross-modal learning, depth feature propagation",

author = "Baian CHEN and Zhilei CHEN and Xiaowei HU and Jun XU and Haoran XIE and Jing QIN and Mingqiang WEI",

note = "This work was supported by the Shenzhen Science and Technology Program (No.JCYJ20220818103401003, No. JCYJ20220530172403007), by the General Program of Natural Science Foundation of Guangdong Province (No. 2022A1515010170), by the Free Exploration of Basic Research Project, Local Science and Technology Development Fund Guided by the Central Government of China (No.2021Szvup060), and the Research Grant entitled {"}Self Supervised Learning for Medical Images{"} (No.871228) and Shenzhen University-Lingnan University Joint Research Programme (SZU-LU006/2122) of Lingnan University, Hong Kong.",

year = "2023",

month = may,

day = "19",

doi = "10.1145/3597612",

language = "English",

journal = "ACM Transactions on Multimedia Computing, Communications and Applications",

issn = "1551-6857",

publisher = "Association for Computing Machinery (ACM)",

}

TY - JOUR

T1 - Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

AU - CHEN, Baian

AU - CHEN, Zhilei

AU - HU, Xiaowei

AU - XU, Jun

AU - XIE, Haoran

AU - QIN, Jing

AU - WEI, Mingqiang

N1 - This work was supported by the Shenzhen Science and Technology Program (No.JCYJ20220818103401003, No. JCYJ20220530172403007), by the General Program of Natural Science Foundation of Guangdong Province (No. 2022A1515010170), by the Free Exploration of Basic Research Project, Local Science and Technology Development Fund Guided by the Central Government of China (No.2021Szvup060), and the Research Grant entitled "Self Supervised Learning for Medical Images" (No.871228) and Shenzhen University-Lingnan University Joint Research Programme (SZU-LU006/2122) of Lingnan University, Hong Kong.

PY - 2023/5/19

Y1 - 2023/5/19

N2 - Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this paper, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately, and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over seventeen methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at: https://github.com/chenbaian-cs/DMPNet.

AB - Exploiting long-range semantic contexts and geometric information is crucial to infer salient objects from RGB and depth features. However, existing methods mainly focus on excavating local features within fixed regions by continuously feeding forward networks. In this paper, we introduce Dynamic Message Propagation (DMP) to dynamically learn context information within more flexible regions. We integrate DMP into a Siamese-based network to process the RGB image and depth map separately, and design a multi-level feature fusion module to explore cross-level information between refined RGB and depth features. Extensive experiments show clear improvements of our method over seventeen methods on six benchmark datasets for RGB-D salient object detection (SOD). Additionally, our method outperforms its competitors for the video SOD task. Code is available at: https://github.com/chenbaian-cs/DMPNet.

KW - RGB-D salient object detection

KW - dynamic message propagation

KW - cross-modal learning

KW - depth feature propagation

U2 - 10.1145/3597612

DO - 10.1145/3597612

M3 - Journal Article (refereed)

SN - 1551-6857

JO - ACM Transactions on Multimedia Computing, Communications and Applications

JF - ACM Transactions on Multimedia Computing, Communications and Applications

ER -

Dynamic Message Propagation Network for RGB-D and Video Salient Object Detection

Abstract

Bibliographical note

Keywords

Access to Document

Fingerprint

Projects

Developing Diagnosis System based on Self-Supervised Adversarial Learning for Babies with Congenital Heart Defects

Self-Supervised Learning for Medical Images

Cite this