Modeling Inner- and Cross-Task Contrastive Relations for Continual Image Classification

Yuxuan LUO, Runmin CONG, Xialei LIU, Horace Ho Shing IP, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review


Existing continual image classification methods demonstrate that samples from all sequences of continual classification tasks contain common (task-invariant) features and class-specific (task-variant) features that can be decoupled for classification tasks. However, the existing feature decomposition strategies only focus on individual tasks while neglecting the essential cues that the relationship between different tasks can provide, thereby hindering the improvement of continual image classification results. To address this issue, we propose an Adversarial Contrastive Continual Learning (ACCL) method that decouples task-invariant and task-variant features by constructing all-round, multi-level contrasts on sample pairs within individual tasks or from different tasks. Specifically, three constraints on the distribution of task-invariant and task-variant features are included, i.e. , task-invariant features across different tasks should remain consistent, task-variant features should exhibit differences, and task-invariant and task-variant features should differ from each other. At the same time, we also design an effective contrastive replay strategy to make full use of the replay samples to participate in the construction of sample pairs, further alleviating the forgetting problem, and modeling cross-task relationships. Through extensive experiments on continual image classification tasks on CIFAR100, MiniImageNet and TinyImageNet, we show the superiority of our proposed strategy, improving the accuracy and with better visualized outcomes. The code and results can be found from the link of .
Original languageEnglish
Number of pages12
JournalIEEE Transactions on Multimedia
Publication statusE-pub ahead of print - 13 Jun 2024

Bibliographical note

Publisher Copyright:


  • Continuing education
  • Feature extraction
  • Image classification
  • Stability analysis
  • Task analysis
  • Thermal stability
  • Training
  • continual learning
  • contrastive learning
  • feature decomposition
  • image classification


Dive into the research topics of 'Modeling Inner- and Cross-Task Contrastive Relations for Continual Image Classification'. Together they form a unique fingerprint.

Cite this