Query-Guided Prototype Evolution Network for Few-Shot Segmentation

Runmin CONG, Hang XIONG, Jinpeng CHEN*, Wei ZHANG, Qingming HUANG, Yao ZHAO

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

Previous Few-Shot Segmentation (FSS) approaches exclusively utilize support features for prototype generation, neglecting the specific requirements of the query. To address this, we present the Query-guided Prototype Evolution Network (QPENet), a new method that integrates query features into the generation process of foreground and background prototypes, thereby yielding customized prototypes attuned to specific queries. The evolution of the foreground prototype is accomplished through a support-query-support iterative process involving two new modules: Pseudo-prototype Generation (PPG) and Dual Prototype Evolution (DPE). The PPG module employs support features to create an initial prototype for the preliminary segmentation of the query image, resulting in a pseudo-prototype reflecting the unique needs of the current query. Subsequently, the DPE module performs reverse segmentation on support images using this pseudo-prototype, leading to the generation of evolved prototypes, which can be considered as custom solutions. As for the background prototype, the evolution begins with a global background prototype that represents the generalized features of all training images. We also design a Global Background Cleansing (GBC) module to eliminate potential adverse components mirroring the characteristics of the current foreground class. Experimental results on the PASCAL-5i and COCO-20i datasets attest to the substantial enhancements achieved by QPENet over prevailing state-of-the-art techniques, underscoring the validity of our ideas.

Original languageEnglish
Pages (from-to)6501-6512
Number of pages12
JournalIEEE Transactions on Multimedia
Volume26
Early online date11 Jan 2024
DOIs
Publication statusPublished - 2024
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

  • few-shot learning
  • Few-shot segmentation
  • prototype generation
  • semantic segmentation

Fingerprint

Dive into the research topics of 'Query-Guided Prototype Evolution Network for Few-Shot Segmentation'. Together they form a unique fingerprint.

Cite this