Perceptual Quality Analysis in Deep Domains Using Structure Separation and High-Order Moments

Weizhi XIAN, Mingliang ZHOU*, Bin FANG, Tao XIANG, Weijia JIA, Bin CHEN

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

1 Citation (Scopus)

Abstract

Images are composed of 'things' (i.e., structured objects) and 'stuff' (i.e., textured surfaces), which have completely different effects on the human visual system (HVS). A good image quality assessment (IQA) method should fully consider the visual salience effects of image structures and the masking effects of image textures. In this article, we propose a perceptual quality analysis model using structure separation and high-order moments (SSHMPQA) in the deep domain. First, we use a total variation (TV) model to separate the perceptual structures in images from their deep feature maps, thereby maintaining meaningful object shapes with texture suppression and defining perceptual structure-aware distances in the deep domain. Then, we use the first- to fourth-order moments to calculate the mean, skewness and kurtosis of the probability distributions of the deep features. On this basis, we define a perceptual texture-aware distance in the deep domain. We then formulate the final model by solving a well-defined perceptual optimization problem. The proposed SSHMPQA model has good interpretability and is data-driven; moreover, the model does not require a complex and long training process because the optimization problem is convex and has an exact analytical solution. To verify the effectiveness of our model, comprehensive experiments are conducted. The experimental results show that the proposed model is superior to other state-of-the-art traditional and deep learning-based full-reference (FR) IQA methods.

Original languageEnglish
Pages (from-to)2219-2234
Number of pages16
JournalIEEE Transactions on Multimedia
Volume26
Early online date12 Jul 2023
DOIs
Publication statusPublished - 2024
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 1999-2012 IEEE.

Keywords

  • Convex optimization
  • deep learning
  • high-order moments
  • perceptual quality
  • probability distribution
  • structure representations

Fingerprint

Dive into the research topics of 'Perceptual Quality Analysis in Deep Domains Using Structure Separation and High-Order Moments'. Together they form a unique fingerprint.

Cite this