Geometry Auxiliary Salient Object Detection for Light Fields via Graph Neural Networks

Qiudan ZHANG, Shiqi WANG, Xu WANG, Zhenhao SUN, Sam KWONG, Jianmin JIANG

Research output: Journal PublicationsJournal Article (refereed)peer-review

10 Citations (Scopus)

Abstract

Light field imaging, originated from the availability of light field capture technology, offers a wide range of applications in the field of computational vision. The capability of predicting salient objects of light fields remains technologically challenging due to its complicated geometry structure. In this paper, we propose a light field salient object detection approach that formulates the geometric coherence among multiple views of light fields as graphs, where the angular/central views represent the nodes and their relations compose the edges. The spatial and disparity correlations between multiple views are effectively explored through multi-scale graph neural networks, enabling the more comprehensive understanding of light field content and more representative and discriminative saliency features generation. Moreover, a multi-scale saliency feature consistency learning module is embedded to enhance the saliency features. Finally, an accurate salient object map is produced for the light field based upon the extracted features. In addition, we establish a new light field salient object detection dataset (CITYU-Lytro) that contains 817 light fields with diverse contents and their corresponding annotations, aiming to further promote the research on light field salient object detection. Quantitative and qualitative experiments demonstrate that the proposed method performs favorably compared with the state-of-the-art methods on the benchmark datasets.
Original languageEnglish
Pages (from-to)7578-7592
JournalIEEE Transactions on Image Processing
Volume30
Early online date1 Sept 2021
DOIs
Publication statusPublished - 2021
Externally publishedYes

Bibliographical note

This work was supported in part by the National Natural Science Foundation of China under Grant 62022002, Grant 61871270, and Grant 61620106008; in part by Shenzhen Natural Science Foundation under Grant JCYJ20200109110410133 and Grant 20200812110350001; in part by the Science, Technology, and Innovation Commission of Shenzhen Municipality under Project 2021Szvup128; in part by Hong Kong Research Grants Council (RGC) Early Career Scheme (ECS) under Grant 21211018; in part by the General Research Fund (GRF) under Grant 11203220; and in part by City University of Hong Kong Applied Research under Grant 9667192.

Keywords

  • graph neural networks
  • Light field
  • salient object detection

Fingerprint

Dive into the research topics of 'Geometry Auxiliary Salient Object Detection for Light Fields via Graph Neural Networks'. Together they form a unique fingerprint.

Cite this