Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression

Huanhua LIU; Yun ZHANG; Huan ZHANG; Chunling FAN; Sam KWONG; C.-C. Jay KUO; Xiaoping FAN

doi:10.1109/TIP.2019.2933743

Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression

Huanhua LIU, Yun ZHANG, Huan ZHANG, Chunling FAN, Sam KWONG, C.-C. Jay KUO, Xiaoping FAN

Research output: Journal Publications › Journal Article (refereed) › peer-review

75 Citations (Scopus)

Abstract

Picture Wise Just Noticeable Difference (PW-JND), which accounts for the minimum difference of a picture that human visual system can perceive, can be widely used in perception-oriented image and video processing. However, the conventional Just Noticeable Difference (JND) models calculate the JND threshold for each pixel or sub-band separately, which may not reflect the total masking effect of a picture accurately. In this paper, we propose a deep learning based PW-JND prediction model for image compression. Firstly, we formulate the task of predicting PW-JND as a multi-class classification problem, and propose a framework to transform the multi-class classification problem to a binary classification problem solved by just one binary classifier. Secondly, we construct a deep learning based binary classifier named perceptually lossy/lossless predictor which can predict whether an image is perceptually lossy to another or not. Finally, we propose a sliding window based search strategy to predict PW-JND based on the prediction results of the perceptually lossy/lossless predictor. Experimental results show that the mean accuracy of the perceptually lossy/lossless predictor reaches 92%, and the absolute prediction error of the proposed PW-JND model is 0.79 dB on average, which show the superiority of the proposed PW-JND model to the conventional JND models.

Original language	English
Pages (from-to)	641-656
Journal	IEEE Transactions on Image Processing
Volume	29
Early online date	13 Aug 2019
DOIs	https://doi.org/10.1109/TIP.2019.2933743
Publication status	Published - 2020
Externally published	Yes

Bibliographical note

This work was supported in part by the National Natural Science Foundation of China under Grant 61871372, in part by the Guangdong Natural Science Foundation for Distinguished Young Scholar under Grant 2016A030306022, in part by the Key Project for Guangdong Provincial Science and Technology Development under Grant 2017B010110014, in part by the Shenzhen International Collaborative Research Project under Grant GJHZ20170314155404913, in part by the Shenzhen Science and Technology Program under Grant JCYJ20170811160212033, in part by the Guangdong International Science and Technology Cooperative Research Project under Grant 2018A050506063, in part by the Membership of Youth Innovation Promotion Association, Chinese Academy of Sciences under Grant 2018392, and in part by the Shenzhen Science and Technology Plan Project under Grant JCYJ20180507183823045.

Keywords

convolutional neural network
image quality assessment
Just noticeable distortion
visual perception

Access to Document

10.1109/TIP.2019.2933743

Cite this

@article{b2d9bb78a48348c6bcc241047cd7fb3b,

title = "Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression",

abstract = "Picture Wise Just Noticeable Difference (PW-JND), which accounts for the minimum difference of a picture that human visual system can perceive, can be widely used in perception-oriented image and video processing. However, the conventional Just Noticeable Difference (JND) models calculate the JND threshold for each pixel or sub-band separately, which may not reflect the total masking effect of a picture accurately. In this paper, we propose a deep learning based PW-JND prediction model for image compression. Firstly, we formulate the task of predicting PW-JND as a multi-class classification problem, and propose a framework to transform the multi-class classification problem to a binary classification problem solved by just one binary classifier. Secondly, we construct a deep learning based binary classifier named perceptually lossy/lossless predictor which can predict whether an image is perceptually lossy to another or not. Finally, we propose a sliding window based search strategy to predict PW-JND based on the prediction results of the perceptually lossy/lossless predictor. Experimental results show that the mean accuracy of the perceptually lossy/lossless predictor reaches 92%, and the absolute prediction error of the proposed PW-JND model is 0.79 dB on average, which show the superiority of the proposed PW-JND model to the conventional JND models.",

keywords = "convolutional neural network, image quality assessment, Just noticeable distortion, visual perception",

author = "Huanhua LIU and Yun ZHANG and Huan ZHANG and Chunling FAN and Sam KWONG and KUO, {C.-C. Jay} and Xiaoping FAN",

note = "This work was supported in part by the National Natural Science Foundation of China under Grant 61871372, in part by the Guangdong Natural Science Foundation for Distinguished Young Scholar under Grant 2016A030306022, in part by the Key Project for Guangdong Provincial Science and Technology Development under Grant 2017B010110014, in part by the Shenzhen International Collaborative Research Project under Grant GJHZ20170314155404913, in part by the Shenzhen Science and Technology Program under Grant JCYJ20170811160212033, in part by the Guangdong International Science and Technology Cooperative Research Project under Grant 2018A050506063, in part by the Membership of Youth Innovation Promotion Association, Chinese Academy of Sciences under Grant 2018392, and in part by the Shenzhen Science and Technology Plan Project under Grant JCYJ20180507183823045.",

year = "2020",

doi = "10.1109/TIP.2019.2933743",

language = "English",

volume = "29",

pages = "641--656",

journal = "IEEE Transactions on Image Processing",

issn = "1057-7149",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - JOUR

T1 - Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression

AU - LIU, Huanhua

AU - ZHANG, Yun

AU - ZHANG, Huan

AU - FAN, Chunling

AU - KWONG, Sam

AU - KUO, C.-C. Jay

AU - FAN, Xiaoping

N1 - This work was supported in part by the National Natural Science Foundation of China under Grant 61871372, in part by the Guangdong Natural Science Foundation for Distinguished Young Scholar under Grant 2016A030306022, in part by the Key Project for Guangdong Provincial Science and Technology Development under Grant 2017B010110014, in part by the Shenzhen International Collaborative Research Project under Grant GJHZ20170314155404913, in part by the Shenzhen Science and Technology Program under Grant JCYJ20170811160212033, in part by the Guangdong International Science and Technology Cooperative Research Project under Grant 2018A050506063, in part by the Membership of Youth Innovation Promotion Association, Chinese Academy of Sciences under Grant 2018392, and in part by the Shenzhen Science and Technology Plan Project under Grant JCYJ20180507183823045.

PY - 2020

Y1 - 2020

N2 - Picture Wise Just Noticeable Difference (PW-JND), which accounts for the minimum difference of a picture that human visual system can perceive, can be widely used in perception-oriented image and video processing. However, the conventional Just Noticeable Difference (JND) models calculate the JND threshold for each pixel or sub-band separately, which may not reflect the total masking effect of a picture accurately. In this paper, we propose a deep learning based PW-JND prediction model for image compression. Firstly, we formulate the task of predicting PW-JND as a multi-class classification problem, and propose a framework to transform the multi-class classification problem to a binary classification problem solved by just one binary classifier. Secondly, we construct a deep learning based binary classifier named perceptually lossy/lossless predictor which can predict whether an image is perceptually lossy to another or not. Finally, we propose a sliding window based search strategy to predict PW-JND based on the prediction results of the perceptually lossy/lossless predictor. Experimental results show that the mean accuracy of the perceptually lossy/lossless predictor reaches 92%, and the absolute prediction error of the proposed PW-JND model is 0.79 dB on average, which show the superiority of the proposed PW-JND model to the conventional JND models.

AB - Picture Wise Just Noticeable Difference (PW-JND), which accounts for the minimum difference of a picture that human visual system can perceive, can be widely used in perception-oriented image and video processing. However, the conventional Just Noticeable Difference (JND) models calculate the JND threshold for each pixel or sub-band separately, which may not reflect the total masking effect of a picture accurately. In this paper, we propose a deep learning based PW-JND prediction model for image compression. Firstly, we formulate the task of predicting PW-JND as a multi-class classification problem, and propose a framework to transform the multi-class classification problem to a binary classification problem solved by just one binary classifier. Secondly, we construct a deep learning based binary classifier named perceptually lossy/lossless predictor which can predict whether an image is perceptually lossy to another or not. Finally, we propose a sliding window based search strategy to predict PW-JND based on the prediction results of the perceptually lossy/lossless predictor. Experimental results show that the mean accuracy of the perceptually lossy/lossless predictor reaches 92%, and the absolute prediction error of the proposed PW-JND model is 0.79 dB on average, which show the superiority of the proposed PW-JND model to the conventional JND models.

KW - convolutional neural network

KW - image quality assessment

KW - Just noticeable distortion

KW - visual perception

UR - http://www.scopus.com/inward/record.url?scp=85072751707&partnerID=8YFLogxK

U2 - 10.1109/TIP.2019.2933743

DO - 10.1109/TIP.2019.2933743

M3 - Journal Article (refereed)

SN - 1057-7149

VL - 29

SP - 641

EP - 656

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

ER -

Deep Learning-Based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this