No-reference image quality assessment by using convolutional neural networks via object detection

Convolutional neural networks (CNNs) have been widely applied in the image quality assessment (IQA) field, but the size of the IQA databases severely limits the performance of the CNN-based IQA models. The most popular method to extend the size of database in previous works is to resize the images into patches. However, human visual system (HVS) can only perceive the qualities of objects in an image rather than the qualities of patches in it. Motivated by this fact, we propose a CNN-based algorithm for no-reference image quality assessment (NR-IQA) based on object detection. The network has three parts: an object detector, an image quality prediction network, and a self-correction measurement (SCM) network. First, we detect objects from input image by the object detector. Second, a ResNet-18 network is applied to extract features of the input image and a fully connected (FC) layer is followed to estimate image quality. Third, another ResNet-18 network is used to extract features of both the images and its detected objects, where the features of the objects are concatenated to the features of the image. Then, another FC layer is followed to compute the correction value of each object. Finally, the predicted image quality is amended by the SCM values. Experimental results demonstrate that the proposed NR-IQA model has state-of-the-art performance. In addition, cross-database evaluation indicates the great generalization ability of the proposed model.