Machine Learning-Based Coding Unit Depth Decisions for Flexible Complexity Allocation in High Efficiency Video Coding

Yun ZHANG, Sam KWONG, Xu WANG, Hui YUAN, Zhaoqing PAN, Long XU

Research output: Journal PublicationsJournal Article (refereed)peer-review

174 Citations (Scopus)


In this paper, we propose a machine learning-based fast coding unit (CU) depth decision method for High Efficiency Video Coding (HEVC), which optimizes the complexity allocation at CU level with given rate-distortion (RD) cost constraints. First, we analyze quad-tree CU depth decision process in HEVC and model it as a three-level of hierarchical binary decision problem. Second, a flexible CU depth decision structure is presented, which allows the performances of each CU depth decision be smoothly transferred between the coding complexity and RD performance. Then, a three-output joint classifier consists of multiple binary classifiers with different parameters is designed to control the risk of false prediction. Finally, a sophisticated RD-complexity model is derived to determine the optimal parameters for the joint classifier, which is capable of minimizing the complexity in each CU depth at given RD degradation constraints. Comparative experiments over various sequences show that the proposed CU depth decision algorithm can reduce the computational complexity from 28.82% to 70.93%, and 51.45% on average when compared with the original HEVC test model. The Bjontegaard delta peak signal-to-noise ratio and Bjontegaard delta bit rate are -0.061 dB and 1.98% on average, which is negligible. The overall performance of the proposed algorithm outperforms those of the state-of-the-art schemes.
Original languageEnglish
Pages (from-to)2225-2238
JournalIEEE Transactions on Image Processing
Issue number7
Early online date27 Mar 2015
Publication statusPublished - Jul 2015
Externally publishedYes


  • Coding Unit
  • High Efficiency Video Coding
  • Machine Learning
  • Support Vector Machine


Dive into the research topics of 'Machine Learning-Based Coding Unit Depth Decisions for Flexible Complexity Allocation in High Efficiency Video Coding'. Together they form a unique fingerprint.

Cite this