Enhanced Motion Compensation for Deep Video Compression

Haifeng GUO, Sam KWONG, Chuanmin JIA, Shiqi WANG

Research output: Journal PublicationsJournal Article (refereed)peer-review

1 Citation (Scopus)

Abstract

Most of the existing deep learning-based video compression frameworks rely on motion estimation and compensation. However, the artifacts of the warped frames after motion compensation, which propagate the errors to the next frame, limit the video coding performance. In this work, we propose enhanced motion compensation for reduced error propagation in deep video compression. More specifically, we incorporate the designed convolutional neural network into Open DVC as the motion compensation enhancement network to remove noise in the predicted frame. With the enhanced frame, we jointly optimize the whole framework with a single loss function by considering the trade-off between bit cost and frame quality. Experiments show that the proposed enhanced motion compensation model reduces error propagation within a group of frames. Compared with Open DVC, our model can achieve 8.94% bit savings on average for standard test videos in terms of PSNR. Regarding MS-SSIM, our model outperforms Open DVC with 5.67% bit rate savings.

Original languageEnglish
Pages (from-to)673-677
Number of pages5
JournalIEEE Signal Processing Letters
Volume30
Early online date17 May 2023
DOIs
Publication statusPublished - 2023
Externally publishedYes

Bibliographical note

Funding Information:
This work was supported in part by the Ministry of Science and Technology of China through Key Project of Science and Technology Innovation 2030 under Grant 2018AAA0101301, in part by Hong Kong Innovation and Technology Commission(InnoHK Project CIMDA), and in part by Hong Kong GRFRGC General Research Fund underGrants 11209819 (CityU 9042816) and 11203820 (9042598).

Publisher Copyright:
© 1994-2012 IEEE.

Keywords

  • convolutional neural network
  • Deep video compression
  • enhanced motion compensation

Fingerprint

Dive into the research topics of 'Enhanced Motion Compensation for Deep Video Compression'. Together they form a unique fingerprint.

Cite this