Rethinking Semantic Image Compression : Scalable Representation with Cross-modality Transfer

Pingping ZHANG, Shiqi WANG, Meng WANG, Jiguo LI, Xu WANG, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

6 Citations (Scopus)

Abstract

This article proposes the scalable cross-modality compression (SCMC) paradigm, in which the image compression problem is further cast into a representation task by hierarchically sketching the image with different modalities. Herein, we adopt the conceptual organization philosophy to model the overwhelmingly complicated visual patterns, based upon the semantic, structure, and signal level representation accounting for different tasks. The SCMC paradigm that incorporates the representation at different granularities supports diverse application scenarios, such as high-level semantic communication and low-level image reconstruction. The decoder, which enables the recovery of the visual information, benefits from the scalable coding based upon the semantic, structure, and signal layers. Qualitative and quantitative results demonstrate that the SCMC can convey accurate semantic and perceptual information of images, especially at low bitrates, and promising rate-distortion performance has been achieved compared to state-of-the-art methods. The code will be available online https://github.com/ppingzhang/SCMC.

Original languageEnglish
Pages (from-to)4441-4445
Number of pages5
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume33
Issue number8
Early online date31 Jan 2023
DOIs
Publication statusPublished - 1 Aug 2023
Externally publishedYes

Bibliographical note

Funding Information:
This work was supported in part by the National Natural Science Foundation of China under Grant 62022002 and Grant 61871270, in part by the Shenzhen Science and Technology Program under Project JCYJ20220530140816037, in part by the Shenzhen Natural Science Foundation under Grant JCYJ20200109110410133.

Publisher Copyright:
© 1991-2012 IEEE.

Keywords

  • cross-modality
  • Data mining
  • Decoding
  • Feature extraction
  • Image coding
  • Image reconstruction
  • scalable coding
  • Semantic image compression
  • Semantics
  • Visualization

Fingerprint

Dive into the research topics of 'Rethinking Semantic Image Compression : Scalable Representation with Cross-modality Transfer'. Together they form a unique fingerprint.

Cite this