A Survey on Perceptually Optimized Video Coding

Yun ZHANG, Linwei ZHU, Gangyi JIANG, Sam KWONG, C.-C. Jay KUO

Research output: Journal PublicationsJournal Article (refereed)peer-review

3 Citations (Scopus)

Abstract

To provide users with more realistic visual experiences, videos are developing in the trends of Ultra High Definition (UHD), High Frame Rate (HFR), High Dynamic Range (HDR), Wide Color Gammut (WCG), and high clarity. However, the data amount of videos increases exponentially, which requires high efficiency video compression for storage and network transmission. Perceptually optimized video coding aims to maximize compression efficiency by exploiting visual redundancies. In this article, we present a broad and systematic survey on perceptually optimized video coding. Firstly, we present problem formulation and framework of the perceptually optimized video coding, which includes visual perception modeling, visual quality assessment, and perceptual video coding optimization. Secondly, recent advances on visual factors, computational perceptual models, and quality assessment models are presented. Thirdly, we review perceptual video coding optimizations from four key aspects, including perceptually optimized bit allocation, rate-distortion optimization, transform and quantization, and filtering and enhancement. In each part, problem formulation, working flow, recent advances, advantages, and challenges are presented. Fourthly, perceptual coding performances of the latest coding standards and tools are experimentally analyzed. Finally, challenging issues and future opportunities are identified.

Original languageEnglish
Article number3571727
JournalACM Computing Surveys
Volume55
Issue number12
Early online date2 Mar 2023
DOIs
Publication statusPublished - 2 Mar 2023
Externally publishedYes

Bibliographical note

Funding Information:
This work was supported in part by the National Natural Science Foundation of China under Grants No. 62172400, No. 61901459, and No. 62271276; in part by the Shenzhen Science and Technology Program under Grant No. JCYJ20200109110410133; in part by the Guangdong Basic and Applied Basic Research Foundation under Grant No. 2022A1515011351; in part by the CAS President’s International Fellowship Initiative (PIFI) under Grant No. 2022VTA0005; in part by the Hong Kong Innovation and Technology Commission (InnoHK Project CIMDA); and in part by the Hong Kong GRF-RGC General Research Fund under Grants No. 11209819 (CityU 9042816) and No. 11203820 (CityU 9042598).

Publisher Copyright:
© 2023 Association for Computing Machinery.

Keywords

  • Additional Key Words and PhrasesPerceptual video coding
  • human visual system
  • just noticeable difference
  • quality assessment
  • rate-distortion optimization
  • visual attention

Fingerprint

Dive into the research topics of 'A Survey on Perceptually Optimized Video Coding'. Together they form a unique fingerprint.

Cite this