Neural Network Based Rate Control for Versatile Video Coding

Yunhao MAO, Meng WANG, Zhangkai NI, Shiqi WANG, Sam KWONG

Research output: Journal PublicationsJournal Article (refereed)peer-review

2 Citations (Scopus)


In this work, we propose a neural network based rate control algorithm for Versatile Video Coding (VVC). The proposed method relies on the modeling of the Rate-Quantization (R-Q) and Distortion-Quantization (D-Q) relationships in a data driven manner based upon the characteristics of prediction residuals. In particular, a pre-analysis framework is adopted, in an effort to obtain the prediction residuals which govern the Rate-Distortion (R-D) behaviors. By inferring from the prediction residuals with deep neural networks, the Coding Tree Unit (CTU) level R-Q and D-Q model parameters are derived, which could efficiently guide the optimal bit allocation. Subsequently, the coding parameters, including Quantization Parameter (QP) and λ , at both frame and CTU levels, are obtained according to allocated bit-rates. We implement the proposed rate control algorithm on VVC Test Model (VTM-13.0). Experimental results exhibit that the proposed rate control algorithm achieves 0.77% BD-Rate savings under Low Delay B (LDB) configurations when compared to the default rate control algorithm used in VTM-13.0. For Random Access (RA) configurations, 1.77% BD-Rate savings can be observed. Furthermore, with better bit-rate estimation, more stable buffer status can be observed, further demonstrating the advantages of the proposed rate control method.

Original languageEnglish
Article number10
Pages (from-to)6072-6085
Number of pages14
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number10
Early online date27 Mar 2023
Publication statusPublished - Oct 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2023 IEEE.


  • Bit rate
  • Convolutional neural networks
  • distortion model
  • Encoding
  • Neural networks
  • Predictive models
  • Quantization (signal)
  • rate control
  • rate model
  • Resource management
  • Versatile video coding


Dive into the research topics of 'Neural Network Based Rate Control for Versatile Video Coding'. Together they form a unique fingerprint.

Cite this