Deep Video Stabilization via Robust Homography Estimation

Weiqing YAN*, Yiqiu SUN, Wujie ZHOU*, Zhaowei LIU, Runmin CONG

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

2 Citations (Scopus)

Abstract

Video stabilization can improve the visual quality of videos that have been captured on mobile devices or other handheld cameras, which are more prone to shaking and motion artifacts. Most of the existing deep video stabilization methods adopts optical flow-based, which produce artifacts and distortions caused by pixel-level warping and enquire expensive computation time. In this letter, we present a novel unsupervised deep video stabilization approach that addresses the influence of moving objects on video stabilization through robust homography estimation. Specifically, we design a foreground mask estimation module as a preprocessing step using a pre-trained semantic segmentation guided method to distinguish the foreground and background regions, enabling us to estimate camera motion via analyzing the background motion. Additionally, we design a low-level confidence feature extraction module to improve motion alignment loss and ensure robust motion estimation. By integrating the learned low-level confidence features with the foreground mask, we can then design a motion estimation module that captures the consistent spatial correspondence between frames through local and global feature extraction. At last, the learnt robust homography is leveraged to stabilize videos. Our method outperforms related state-of-the-art approaches in both quality and quantity on three public benchmarks while remaining computationally efficient.

Original languageEnglish
Pages (from-to)1602-1606
Number of pages5
JournalIEEE Signal Processing Letters
Volume30
Early online date1 Nov 2023
DOIs
Publication statusPublished - 2023
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 1994-2012 IEEE.

Keywords

  • homography estimation
  • local and global feature extraction
  • Video stabilization

Fingerprint

Dive into the research topics of 'Deep Video Stabilization via Robust Homography Estimation'. Together they form a unique fingerprint.

Cite this