No-reference image quality assessment via multibranch convolutional neural networks

Zhaoqing PAN, Feng YUAN, Xu WANG, Long XU, Xiao SHAO, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

14 Citations (Scopus)

Abstract

No-reference image quality assessment (NR-IQA) aims to evaluate image quality without using the original reference images. Since the early NR-IQA methods based on distortion types were only applicable to specific distortion scenarios, and lack of practicality, it is challenging to designing a universal NR-IQA method. In this article, a multibranch convolutional neural network (MB-CNN) based NR-IQA method is proposed, which includes a spatial-domain feature extractor, a gradient-domain feature extractor, and a weight mechanism. The spatial-domain feature extractor aims to extract the distortion features from the spatial domain. The gradient-domain feature extractor is used to guide the spatial-domain feature extractor to pay more attention to the distortions of the structure information. Particularly, the spatial-domain feature extractor uses the hierarchical feature merge module to realize multiscale feature representation, and the gradient-domain feature extractor uses pyramidal convolution to extract the multiscale structure information of the distorted image. In addition, a position vector is proposed to build the weight mechanism by considering the position relationships between patches and its entire image for improving the final prediction performance. We conduct the experiments on five representative databases: LIVE, TID2013, CSIQ, LIVE MD and Waterloo Exploration Database, and the experimental results show that the proposed NR-IQA method achieves the state-of-the-art performance, which demonstrate the effectiveness of our proposed NR-IQA method. The code ofthe proposed MB-CNN will be released at https://github.com/NUIST-Videocoding/MB-CNN.
Original languageEnglish
Pages (from-to)148-160
Number of pages13
JournalIEEE Transactions on Artificial Intelligence
Volume4
Issue number1
Early online date27 Jan 2022
DOIs
Publication statusPublished - Feb 2023
Externally publishedYes

Bibliographical note

Funding Information:
Thisworkwas supported in part by the National Natural Science Foundation of China under Grant 61971232 and Grant 61871270, in part by the Natural Science Foundation of Jiangsu Province of China under Grant BK20201391, and in part by the Shenzhen Natural Science Foundation under Grant JCYJ20200109110410133 and Grant 20200812110350001.

Publisher Copyright:
© 2020 IEEE.

Keywords

  • Hierarchical feature merge module (HFMM)
  • multibranch CNN
  • no-reference image quality assessment (NR-IQA)
  • position features

Fingerprint

Dive into the research topics of 'No-reference image quality assessment via multibranch convolutional neural networks'. Together they form a unique fingerprint.

Cite this