Multilevel Similarity-Aware Deep Metric Learning for Fine-Grained Image Retrieval

Congcong DUAN, Yong FENG*, Mingliang ZHOU*, Xiancai XIONG, Yongheng WONG, Baohua QIANG, Weijia JIA

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

1 Citation (Scopus)


Fast and accurate image retrieval is an important and challenging task in massive image data scenarios. As the core technology of image retrieval tasks, deep metric learning aims at learning effective embedding representations that possess two properties among data points: positive concentrated and negative separated. In this work, we propose a multilevel similarity-aware method based on deep local descriptors for deep metric learning. We take the rich interclass similarity relationship based on the deep local invariant descriptors from the data into account to optimize sampling strategies for mining informative samples. The method dynamically adjusts the margin between data points to better match the true similarity relationship between classes. Specifically, for images in a batch, we first obtain deep local descriptors and calculate the similarity matrix of the channel, pixel, and spatial levels. Then, depending on the calculated comprehensive similarity matrix, we propose a multilevel similarity-aware loss function through the deviation between pairwise distance and violate margin to make full use of informative samples. The experimental results demonstrate that our proposed method outperforms other state-of-the-art methods in terms of fine-grained image retrieval and clustering tasks.

Original languageEnglish
Pages (from-to)9173-9182
Number of pages10
JournalIEEE Transactions on Industrial Informatics
Issue number8
Publication statusPublished - Aug 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2005-2012 IEEE.


  • Deep local descriptors
  • deep metric learning (DML)
  • fine-grained image retrieval
  • similarity metric


Dive into the research topics of 'Multilevel Similarity-Aware Deep Metric Learning for Fine-Grained Image Retrieval'. Together they form a unique fingerprint.

Cite this