Image Quality Assessment: Unifying Structure and Texture Similarity
Event date:  November 2, 2020 4:00 PM  to November 2, 0202 6:06 PM

Abstract

Objective measures of image quality generally operate by making local comparisons of pixels of a „degraded" image to those of the original. Relative to human observers, these measures are overly sensitive to resampling of texture regions (e.g., replacing one patch of grass with another). Here we develop the first full-reference image quality model with explicit tolerance to texture resampling. Using a convolutional neural network, we construct an injective and differentiable function that transforms images to a multi-scale overcomplete representation. We empirically show that the spatial averages of the feature maps in this representation capture texture appearance, in that they provide a set of sufficient statistical constraints to synthesize a wide variety of texture patterns. We then describe an image quality method that combines correlation of these spatial averages („texture similarity") with correlation of the feature maps („ structure similarity"). The parameters of the proposed measure are jointly optimized to match human ratings of image quality, while minimizing the reported distances between subimages cropped from the same texture images. Experiments show that the optimized method explains human perceptual scores, both on conventional image quality databases, as well as on texture databases. The measure also offers competitive performance on related tasks such as texture classification and retrieval. Finally, we show that our method is relatively insensitive to geometric transformations (e.g., translation and dilation), without use of any specialized training or data augmentation.


About the Speaker

Kede Ma is an Assistant Professor with the Department of Computer Science at City University of Hong Kong (CityU). He received the B.E. degree from the University of Science and Technology of China (USTC) in 2012, the M.Sc. and Ph.D. degrees from the University of Waterloo, in 2014 and 2017, respectively. Prior to joining CityU, he was a Research Associate with Howard Hughes Medical Institute and New York University, from 2018 to 2019. Dr. Ma’s research interests span perceptual image processing, computational vision, computational photography, and multimedia forensics. In recent years, he primarily focused on image/video quality assessment, based on which better image/video processing algorithms that are much “healthier” for our visual systems can be created.


About the Series

The Lecture Series is organized every winter term and consist of talks with international speakers. The speakers are experts from various fields and professions. The talks are aimed at covering the whole spectrum of visual computing and at discussing the relevance of quantification. This years lecture series will be held entirely online.


This is an online event only.

Registration: If you would like to attend, please email Leonel Merino to register.