Stereo superpixel segmentation via decoupled dynamic spatial-embedding fusion network

Hua LI, Junyan LIANG, Ruiqi WU, Runmin CONG, Wenhui WU, Sam Tak Wu KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

1 Citation (Scopus)

Abstract

Stereo superpixel segmentation aims at grouping the discretizing pixels into perceptual regions through left and right views more collaboratively and efficiently. Existing superpixel segmentation algorithms mostly utilize color and spatial features as input, which may impose strong constraints on spatial information while utilizing the disparity information in terms of stereo image pairs. To alleviate this issue, we propose a stereo superpixel segmentation method with a decoupling mechanism of spatial information in this work. To decouple stereo disparity information and spatial information, the spatial information is temporarily removed before fusing the features of stereo image pairs, and a decoupled stereo fusion module (DSFM) is designed to handle the stereo features alignment as well as occlusion problems. Moreover, since the spatial information is vital to superpixel segmentation, we further design a dynamic spatiality embedding module (DSEM) to re-add spatial information, and the weights of spatial information will be adaptively adjusted through the dynamic fusion (DF) mechanism in DSEM for achieving a finer segmentation. Comprehensive experimental results demonstrate that our method can achieve the state-of-the-art performance on the KITTI2015 and Cityscapes datasets, and also verify the efficiency when applied in salient object detection on NJU2K dataset. The source code will be available publicly after paper is accepted.

Original languageEnglish
Pages (from-to)1-12
Number of pages12
JournalIEEE Transactions on Multimedia
DOIs
Publication statusE-pub ahead of print - 10 Apr 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
IEEE

Keywords

  • Collaboration
  • Computer science
  • Feature extraction
  • Image color analysis
  • Image segmentation
  • Object detection
  • spatiality embedding
  • stereo corresponding capturing
  • Stereo image
  • superpixel segmentation
  • Task analysis

Fingerprint

Dive into the research topics of 'Stereo superpixel segmentation via decoupled dynamic spatial-embedding fusion network'. Together they form a unique fingerprint.

Cite this