Cross-modal identity correlation mining for visible-thermal person re-identification

Sen ZHANG, Zhaowei SHANG*, Mingliang ZHOU*, Yingxin WANG, Guoliang SUN

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

2 Citations (Scopus)

Abstract

Visible-thermal person recognition is a sub problem of image retrieval, which aims to find out the images belonging to the same pedestrian as the current image from the image set of another modality. In this paper, we propose a novel cross-modal identity correlation mining algorithm to mine potential correlation knowledge from the features of visible and thermal modalities. First, aiming at the huge visual differences caused by different imaging mechanisms, we build a correlation-enhanced knowledge transfer module based on cross-modal identity similarity to enhance the feature representation by exchanging identity knowledge between two modalities and then compress it into a shared subspace. Second, in view of different pedestrian posture and camera perspective, we design a symmetric modal-specific feature embedding module to improve the intra-modality feature discrimination, which maps the two modal images to a pair of independent feature subspaces by two fine-grained network branches. The whole algorithm can be trained in an end-to-end manner. Extensive experiments demonstrated that the proposed method outperforms the state-of-the-art methods on SYSU-MM01 and RegDB.

Original languageEnglish
Pages (from-to)39981-39994
Number of pages14
JournalMultimedia Tools and Applications
Volume81
Issue number28
Early online date5 May 2022
DOIs
Publication statusPublished - Nov 2022
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

  • Cross-modal
  • Feature embedding
  • Identity similarity
  • Knowledge transfer
  • Pedestrian reidentification

Fingerprint

Dive into the research topics of 'Cross-modal identity correlation mining for visible-thermal person re-identification'. Together they form a unique fingerprint.

Cite this