Multi-Stage Visual Tracking with Siamese Anchor-Free Proposal Network

Guang HAN, Jinpeng SU, Yaoming LIU, Yuqiu ZHAO, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review


The austere challenge of visual object tracking is to find the target to be tracked in various noise interference and obtain its accurate bounding box coordinates. Recently, the object tracking technology based on the Siamese network has made great breakthroughs, and more and more Siamese network trackers have been proposed with superior performance. They still have some shortcomings. To this end, a new Multi-Stage visual tracking algorithm with Siamese Anchor-Free Proposal Network (MS-SiamAFPN) is proposed in this paper. The algorithm is a three-stage Siamese network tracker composed of Feature Extraction and Fusion (FEF) sub-network, Classification and Regression (CR) sub-network, Validation and Regression (VR) sub-network in series. Firstly, the Anchor-Free Proposal Network (AFPN) module is designed in the CR stage, which can make full use of positive and negative samples for training while reducing neural network parameters. Secondly, aim to achieve better robustness and recognizability in the VR stage, on the one hand, a novel Feature Purification (FP) module is designed, which can automatically select the important channels, and extract the features of irregular regions on the input fusion features, so as to strengthen the representation ability of image features. On the other hand, the target recognition and position regression are regarded as different processing tasks, and the recognition score and position fine-tuning of candidate targets are obtained by newly designing the Dual-Branch Network (DBN) structure, thereby avoiding feature ambiguity. Due to the synergy of the above these innovations, MS-SiamAFPN has obtained a large performance improvement, and achieved SOTA performance in multiple public dataset benchmarks.
Original languageEnglish
Number of pages14
JournalIEEE Transactions on Multimedia
Publication statusE-pub ahead of print - 15 Nov 2021
Externally publishedYes

Bibliographical note

This work was supported in part by the Natural Science Foundation of China NSFC under Grants 61871445 and 61302156 and in part by the Key R & D Foundation Project of Jiangsu Province under Grant BE2016001-4.


  • Anchor-free
  • feature purification
  • object tracking
  • siamese network


Dive into the research topics of 'Multi-Stage Visual Tracking with Siamese Anchor-Free Proposal Network'. Together they form a unique fingerprint.

Cite this