Neural Network Based Rate Control for Versatile Video Coding

Yunhao MAO, Meng WANG, Zhangkai NI, Shiqi WANG, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

8 Citations (Scopus)

Abstract

In this work, we propose a neural network based rate control algorithm for Versatile Video Coding (VVC). The proposed method relies on the modeling of the Rate-Quantization (R-Q) and Distortion-Quantization (D-Q) relationships in a data driven manner based upon the characteristics of prediction residuals. In particular, a pre-analysis framework is adopted, in an effort to obtain the prediction residuals which govern the Rate-Distortion (R-D) behaviors. By inferring from the prediction residuals with deep neural networks, the Coding Tree Unit (CTU) level R-Q and D-Q model parameters are derived, which could efficiently guide the optimal bit allocation. Subsequently, the coding parameters, including Quantization Parameter (QP) and λ , at both frame and CTU levels, are obtained according to allocated bit-rates. We implement the proposed rate control algorithm on VVC Test Model (VTM-13.0). Experimental results exhibit that the proposed rate control algorithm achieves 0.77% BD-Rate savings under Low Delay B (LDB) configurations when compared to the default rate control algorithm used in VTM-13.0. For Random Access (RA) configurations, 1.77% BD-Rate savings can be observed. Furthermore, with better bit-rate estimation, more stable buffer status can be observed, further demonstrating the advantages of the proposed rate control method.

Original languageEnglish
Article number10
Pages (from-to)6072-6085
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Volume33
Issue number10
Early online date27 Mar 2023
DOIs
Publication statusPublished - Oct 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Funding

This work was supported in part by the Hong Kong Innovation and Technology Commission [InnoHK Project Center of Intelligent Multimedia Data Analysis (CIMDA)]; in part by the Hong Kong General Research Fund (GRF)-Research Grant Council (RGC) General Research Fund under Grant 11209819 (CityU 9042816), Grant 11203820 (CityU 9042958), and Grant 11203220 (CityU 9042957); in part by the National Natural Science Foundation of China under Grant 62022002 and Grant 62201387; and in part by the Shanghai Pujiang Program under Grant 22PJ1413300 The authors would like to thank the Associate Editor and anonymous reviewers for their valuable comments that significantly helped them in improving the quality of the paper.

Keywords

  • Bit rate
  • Convolutional neural networks
  • distortion model
  • Encoding
  • Neural networks
  • Predictive models
  • Quantization (signal)
  • rate control
  • rate model
  • Resource management
  • Versatile video coding

Fingerprint

Dive into the research topics of 'Neural Network Based Rate Control for Versatile Video Coding'. Together they form a unique fingerprint.

Cite this