Flexible mode selection and complexity allocation in high efficiency video coding

To improve compression performance, High Efficiency Video Coding (HEVC) employs a quad-tree based block representation, namely Coding Tree Unit (CTU), which can support larger partitions and more coding modes than a traditional macroblock. Despite its high compression efficiency, the number of combinations of coding modes increases dramatically, which results in high computational complexity at the encoder. Here we propose a flexible framework for HEVC coding mode selection, with a user-defined global complexity factor. Based on linear programming, a hierarchical complexity allocation scheme is developed to allocate computational complexities among frames and Coding Units (CUs) to maximize the overall Rate-Distortion (RD) performance. In each CU, with the allocated complexity factor, a mode mapping based approach is employed for coding mode selection. Extensive experiments demonstrate that, with a series of global complexity factors, the proposed model can achieve good trade-offs between computational complexity and RD performance. © 2007-2012 IEEE.