A Multi-Task Collaborative Network for Light Field Salient Object Detection

Qiudan ZHANG, Shiqi WANG, Xu WANG, Zhenhao SUN, Sam KWONG, Jianmin JIANG

Research output: Journal PublicationsJournal Article (refereed)peer-review

41 Citations (Scopus)

Abstract

Being able to predict the salient object is of fundamental importance in image processing and computer vision. With numerous approaches proposed for automatic image and video salient object detection, much less work has been dedicated to detecting and segmenting salient objects from light fields. In this article, based on the intrinsic characteristics of light fields, we carefully explore the complementary coherence among multiple cues including spatial, edge and depth information, and elaborately design a multi-task collaborative network for light field salient object detection. More specifically, the correlation mechanisms among edge detection, depth inference and salient object detection are carefully investigated to facilitate the representative saliency features. We first model the coherence among low-level features and heuristic semantic priors, as well as the edge information. Subsequently, the depth-oriented saliency features are derived from the geometry of light fields, in which the 3D convolution operation is leveraged with powerful representation capability to model the disparity correlations among multiple viewpoint images. Finally, a feature-enhanced salient object generator is developed to integrate these complementary saliency features, leading to the final salient object predictions for light fields. Quantitative and qualitative experiments demonstrate the superiority of our proposed model against the state-of-the-art methods over the public light field salient object detection datasets.
Original languageEnglish
Pages (from-to)1849-1861
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume31
Issue number5
Early online date30 Jul 2020
DOIs
Publication statusPublished - May 2021
Externally publishedYes

Bibliographical note

This work was supported by the Hong Kong RGC Early Career Scheme under Grant 9048122 (CityU 21211018).

Keywords

  • Light field
  • multi-task collaborative network
  • salient object

Fingerprint

Dive into the research topics of 'A Multi-Task Collaborative Network for Light Field Salient Object Detection'. Together they form a unique fingerprint.

Cite this