UAFer: A Unified Model for Class-agnostic Binary Segmentation with Uncertainty-Aware Feature Reassembly

Zihao DONG, Zizhen LIU, Runmin CONG, Tiyu FANG, Xiuli SHAO, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review


Class-agnostic binary segmentation identifies objects that are similar or very different from the complex background, including salient object detection (SOD) and camouflage object detection (COD). Most existing models only focus on a specific type of foreground and background segmentation by employing the global modeling ability of transformers, without explicitly explaining or eliminating the discrepancy between these two different distributions. They also suffer from inefficient local feature learning and inadequate feature aggregation. To make binary segmentation research more accessible and trivially generalized, we introduce a novel unified uncertainty-aware paradigm, called uncertainty-aware feature reassembly (UAFer). Specifically, the Spatial Feature Reassembly (SFR) module is presented to formulate the uncertainty of binary segmentation map as the variance of generalized Bernoulli distribution and entropy from two perspectives. Our transformer-based model is then trained to prioritize regions of higher certainty, obtaining more confident and accurate predictions during the feature upsampling. Moreover, the Channel Feature Reassembly (CFR) with adjacent feature aggregation is designed to facilitate an iterative exploration of channel integrity. This iterative learning process enhances the interaction of neighboring channel features; thus, improving universal object information decoding efficiency. Extensive quantitative and qualitative evaluations demonstrate that our proposed UAFer consistently outperforms the state-of-the-art models across three challenging domains including SOD, COD, and polyp segmentation (POLYP). The implementation codes for our approach will be publicly available at
Original languageEnglish
JournalIEEE Transactions on Circuits and Systems for Video Technology
Publication statusE-pub ahead of print - 20 May 2024

Bibliographical note

Publisher Copyright:


  • Class-agnostic binary segmentation
  • Decoding
  • Feature extraction
  • Feature reassembly
  • Image edge detection
  • Image segmentation
  • Spatial and channel features
  • Task analysis
  • Transformers
  • Uncertainty
  • Uncertainty-aware
  • Unified model


Dive into the research topics of 'UAFer: A Unified Model for Class-agnostic Binary Segmentation with Uncertainty-Aware Feature Reassembly'. Together they form a unique fingerprint.

Cite this