An End-to-End No-Reference Video Quality Assessment Method With Hierarchical Spatiotemporal Feature Representation

Wenhao SHEN, Mingliang ZHOU, Xingran LIAO, Weijia JIA, Tao XIANG, Bin FANG, Zhaowei SHANG

Research output: Journal PublicationsJournal Article (refereed)peer-review

22 Citations (Scopus)

Abstract

In this paper, we propose a deep neural network-based no-reference (NR) video quality assessment (VQA) method with spatiotemporal feature fusion and hierarchical information integration to evaluate the perceptual quality of videos. First, a feature extraction model is proposed by using 2D and 3D convolutional layers to gradually extract spatiotemporal features from raw video clips. Second, we design a hierarchical branching network to fuse multiframe features, and the feature vectors at each hierarchical level are comprehensively considered during the process of network optimization. Finally, these two modules and quality regression are synthesized into an end-to-end architecture. Experimental results obtained on benchmark VQA databases demonstrate the superiority of our method over other state-of-the-art algorithms. The source code is available online.1

Original languageEnglish
Pages (from-to)651-660
Number of pages10
JournalIEEE Transactions on Broadcasting
Volume68
Issue number3
Early online date11 Apr 2022
DOIs
Publication statusPublished - Sept 2022
Externally publishedYes

Bibliographical note

Publisher Copyright:
IEEE

Keywords

  • deep neural network
  • Feature extraction
  • Neural networks
  • Quality assessment
  • spatiotemporal information
  • Spatiotemporal phenomena
  • Streaming media
  • Video quality assessment
  • Video recording
  • Visualization

Fingerprint

Dive into the research topics of 'An End-to-End No-Reference Video Quality Assessment Method With Hierarchical Spatiotemporal Feature Representation'. Together they form a unique fingerprint.

Cite this