U-Shaped Transformer-Based 360-Degree No Reference Image Quality Assessment

Xuekai WEI, Mingliang ZHOU, Sam KWONG

Research output: Book Chapters | Papers in Conference ProceedingsConference paper (refereed)Researchpeer-review

1 Citation (Scopus)


Thanks to creative rendering and display techniques, 360-degree images can provide a more immersive and interactive experience for streaming users. However, such features make the perceptual characteristics of 360-degree images more complex than those of fixed-view images, making it impossible to achieve a comprehensive and accurate image quality assessment (IQA) task using a simple stack of pre-processing, post-processing, compression, and rendering tasks. In order to thoroughly learn global and local features in 360-degree images, reduce the complexity of multichannel neural network models and simplify the training process, this paper proposes a user-Aware joint architecture and an efficient converter dedicated to 360-degree no-reference (NR) IQA. The input of the proposed method is a 360-degree cubic mapping projection (CMP) image. In addition, the proposed 360-degree NRIQA method includes a non-overlapping self-Attentive selection module based on a dominant map and a feature extraction module based on a U-shaped transformer (U-former) to address perceptual region significance and projection distortion. The transformer-based architecture and the weighted averaging technique are jointly used to predict local perceptual quality. Experimental results obtained on widely used databases show that the proposed model outperforms other state-of-The-Art methods in the case of NR 360-degree image quality assessment. In addition, cross-database evaluation and ablation studies demonstrate the intrinsic robustness and generalization of the proposed model.
Original languageEnglish
Title of host publicationProceedings of the 2022 IEEE 21st International Conference on Cognitive Informatics and Cognitive Computing
Publication statusPublished - Dec 2022
Externally publishedYes
Event2022 IEEE 21st International Conference on Cognitive Informatics & Cognitive Computing - , Canada
Duration: 8 Dec 202210 Dec 2022


Conference2022 IEEE 21st International Conference on Cognitive Informatics & Cognitive Computing


  • 360-degree image
  • image quality assessment
  • transformer


Dive into the research topics of 'U-Shaped Transformer-Based 360-Degree No Reference Image Quality Assessment'. Together they form a unique fingerprint.

Cite this