Neural Network Based Rate Control for Versatile Video Coding

Yunhao MAO, Meng WANG, Zhangkai NI, Shiqi WANG, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review


In this work, we propose a neural network based rate control algorithm for Versatile Video Coding (VVC). The proposed method relies on the modeling of the Rate-Quantization (R-Q) and Distortion-Quantization (D-Q) relationships in a data driven manner based upon the characteristics of prediction residuals. In particular, a pre-analysis framework is adopted, in an effort to obtain the prediction residuals which govern the Rate-Distortion (R-D) behaviors. By inferring from the prediction residuals with deep neural networks, the Coding Tree Unit (CTU) level R-Q and D-Q model parameters are derived, which could efficiently guide the optimal bit allocation. Subsequently, the coding parameters, including Quantization Parameter (QP) and λ, at both frame and CTU levels, are obtained according to allocated bit-rates. We implement the proposed rate control algorithm on VVC Test Model (VTM-13.0). Experimental results exhibit that the proposed rate control algorithm achieves 0.77% BD-Rate savings under Low Delay B (LDB) configurations when compared to the default rate control algorithm used in VTM-13.0. For Random Access (RA) configurations, 1.77% BD-Rate savings can be observed. Furthermore, with better bit-rate estimation, more stable buffer status can be observed, further demonstrating the advantages of the proposed rate control method.

Original languageEnglish
Number of pages15
JournalIEEE Transactions on Circuits and Systems for Video Technology
Publication statusE-pub ahead of print - 27 Mar 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:


  • Bit rate
  • Convolutional neural networks
  • distortion model
  • Encoding
  • Neural networks
  • Predictive models
  • Quantization (signal)
  • rate control
  • rate model
  • Resource management
  • Versatile video coding


Dive into the research topics of 'Neural Network Based Rate Control for Versatile Video Coding'. Together they form a unique fingerprint.

Cite this