A Weakly Supervised Learning Framework for Salient Object Detection via Hybrid Labels

Runming CONG, Qi QIN, Chen ZHANG, Qiuping JIANG, Shiqi WANG, Yao ZHAO, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

27 Citations (Scopus)

Abstract

Fully-supervised salient object detection (SOD) methods have made great progress, but such methods often rely on a large number of pixel-level annotations, which are time-consuming and labour-intensive. In this paper, we focus on a new weakly-supervised SOD task under hybrid labels, where the supervision labels include a large number of coarse labels generated by the traditional unsupervised method and a small number of real labels. To address the issues of label noise and quantity imbalance in this task, we design a new pipeline framework with three sophisticated training strategies. In terms of model framework, we decouple the task into label refinement sub-task and salient object detection sub-task, which cooperate with each other and train alternately. Specifically, the R-Net is designed as a two-stream encoder-decoder model equipped with Blender with Guidance and Aggregation Mechanisms (BGA), aiming to rectify the coarse labels for more reliable pseudo-labels, while the S-Net is a replaceable SOD network supervised by the pseudo labels generated by the current R-Net. Note that, we only need to use the trained S-Net for testing. Moreover, in order to guarantee the effectiveness and efficiency of network training, we design three training strategies, including alternate iteration mechanism, group-wise incremental mechanism, and credibility verification mechanism. Experiments on five SOD benchmarks show that our method achieves competitive performance against weakly-supervised/unsupervised methods both qualitatively and quantitatively. The code and results can be found from the link of https://rmcong.github.io/proj Hybrid-Label-SOD.html.
Original languageEnglish
Pages (from-to)534-548
Number of pages15
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume33
Issue number2
Early online date8 Sept 2022
DOIs
Publication statusPublished - Feb 2023
Externally publishedYes

Bibliographical note

Funding Information:
This work was supported in part by the National Key Research and Development Program of China under Grant 2021ZD0112100; in part by the Beijing Nova Program under Grant Z201100006820016; in part by the National Natural Science Foundation of China under Grant 62002014, Grant U1936212, and Grant 62120106009; in part by the Beijing Natural Science Foundation under Grant 4222013; in part by the Hong Kong Innovation and Technology Commission (InnoHK Project CIMDA); in part by the Hong Kong GRF-RGC General Research Fund under Grant 11209819 (CityU 9042816) and Grant 11203820 (CityU 9042598); in part by the Natural Science Foundation of Zhejiang under Grant LR22F020002; in part by the Young Elite Scientist Sponsorship Program by the China Association for Science and Technology under Grant 2020QNRC001; and in part by the CAAI-Huawei MindSpore Open Fund. The work of Runmin Cong was supported in part by the Fundamental Research Funds for the Central Universities under Grant 2022JBMC002.

Publisher Copyright:
© 1991-2012 IEEE.

Keywords

  • blender
  • group-wise incremental mechanism
  • hybrid labels
  • Salient object detection
  • weakly supervised learning

Fingerprint

Dive into the research topics of 'A Weakly Supervised Learning Framework for Salient Object Detection via Hybrid Labels'. Together they form a unique fingerprint.

Cite this