Deep Learning-Based Chroma Prediction for Intra Versatile Video Coding

Linwei ZHU, Yun ZHANG, Shiqi WANG, Sam KWONG, Xin JIN, Yu QIAO

Research output: Journal PublicationsJournal Article (refereed)peer-review

21 Citations (Scopus)


Color images always exhibit a high correlation between luma and chroma components. Cross component linear model (CCLM) has been introduced to exploit such correlation for removing redundancy in the on-going video coding standard, i.e., versatile video coding (VVC). To further improve the coding performance, this paper presents a deep learning based intra chroma prediction method, termed as convolutional neural network based chroma prediction (CNNCP). More specifically, the process of chroma prediction is formulated to produce the colorful version from available information input. CNNCP includes two sub-networks for luma down-sampling and chroma prediction, which are jointly optimized to fully exploit spatial and cross component information. In addition, the outputs of CCLM are adopted as chroma initialization for performance enhancement, and the coding distortion level characterized by quantization parameter is fed into the network to release the negative affect from compression artifacts. To further improve the coding performance, the competition is performed between the conventional chroma prediction and CNNCP in terms of rate-distortion cost with a binary flag signalled. The learned CNNCP is incorporated into both video encoder and decoder. Extensive experimental results demonstrate that the proposed scheme can achieve 4.283%, 3.343%, and 4.634% bit rate savings for luma and two chroma components, compared with the VVC test model version 4.0 (VTM 4.0).
Original languageEnglish
Pages (from-to)3168-3181
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number8
Early online date3 Nov 2020
Publication statusPublished - Aug 2021
Externally publishedYes

Bibliographical note

This work was supported in part by the Shenzhen Science and Technology Program under Grant JCYJ20180507183823045, in part by the Natural Science Foundation of China under Grant 61901459 and Grant 61672443, in part by the China Postdoctoral Science Foundation under Grant 2019M653127, in part by the Guangdong International Science and Technology Cooperative Research Project under Grant 2018A050506063, in part by the Membership of Youth Innovation Promotion Association, Chinese Academy of Sciences under Grant 2018392, in part by the Hong Kong RGC General Research Funds under Grant 9042816 (CityU 11209819), and in part by the Hong Kong RGC Early Career Scheme under Grant 9048122 (CityU 21211018).


  • Chroma prediction
  • Convolutional neural network
  • Deep learning
  • Versatile video coding


Dive into the research topics of 'Deep Learning-Based Chroma Prediction for Intra Versatile Video Coding'. Together they form a unique fingerprint.

Cite this