Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression

Huanhua LIU, Yun ZHANG, Huan ZHANG, Chunling FAN, Sam KWONG, C.-C. Jay KUO, Xiaoping FAN

Research output: Journal PublicationsJournal Article (refereed)peer-review

72 Citations (Scopus)

Abstract

Picture Wise Just Noticeable Difference (PW-JND), which accounts for the minimum difference of a picture that human visual system can perceive, can be widely used in perception-oriented image and video processing. However, the conventional Just Noticeable Difference (JND) models calculate the JND threshold for each pixel or sub-band separately, which may not reflect the total masking effect of a picture accurately. In this paper, we propose a deep learning based PW-JND prediction model for image compression. Firstly, we formulate the task of predicting PW-JND as a multi-class classification problem, and propose a framework to transform the multi-class classification problem to a binary classification problem solved by just one binary classifier. Secondly, we construct a deep learning based binary classifier named perceptually lossy/lossless predictor which can predict whether an image is perceptually lossy to another or not. Finally, we propose a sliding window based search strategy to predict PW-JND based on the prediction results of the perceptually lossy/lossless predictor. Experimental results show that the mean accuracy of the perceptually lossy/lossless predictor reaches 92%, and the absolute prediction error of the proposed PW-JND model is 0.79 dB on average, which show the superiority of the proposed PW-JND model to the conventional JND models.
Original languageEnglish
Pages (from-to)641-656
JournalIEEE Transactions on Image Processing
Volume29
Early online date13 Aug 2019
DOIs
Publication statusPublished - 2020
Externally publishedYes

Bibliographical note

This work was supported in part by the National Natural Science Foundation of China under Grant 61871372, in part by the Guangdong Natural Science Foundation for Distinguished Young Scholar under Grant 2016A030306022, in part by the Key Project for Guangdong Provincial Science and Technology Development under Grant 2017B010110014, in part by the Shenzhen International Collaborative Research Project under Grant GJHZ20170314155404913, in part by the Shenzhen Science and Technology Program under Grant JCYJ20170811160212033, in part by the Guangdong International Science and Technology Cooperative Research Project under Grant 2018A050506063, in part by the Membership of Youth Innovation Promotion Association, Chinese Academy of Sciences under Grant 2018392, and in part by the Shenzhen Science and Technology Plan Project under Grant JCYJ20180507183823045.

Keywords

  • convolutional neural network
  • image quality assessment
  • Just noticeable distortion
  • visual perception

Fingerprint

Dive into the research topics of 'Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression'. Together they form a unique fingerprint.

Cite this