Measuring multivariate subjective image quality for still and video cameras and image processing system components Göte Nyman a , Tuomas Leisti a , Paul Lindroos a , Jenni Radun a , Sini Suomi a , Toni Virtanen a , Jean-Luc Olives b , Joni Oja b & Tero Vuori b a Department of Psychology, University of Helsinki, P.O.Box 9, 00014 Helsinki, Finland; b Nokia Research Center, P.O.Box 407, 00045 Nokia Group, Finland ABSTRACT The subjective quality of an image is a non-linear product of several, simultaneously contributing subjective factors such as the experienced naturalness, colorfulness, lightness, and clarity. We have studied subjective image quality by using a hybrid qualitative/ quantitative method in order to disclose relevant attributes to experienced image quality. We describe our approach in mapping the image quality attribute space in three cases: still studio image, video clips of a talking head and moving objects, and in the use of image processing pipes for 15 still image contents. Naïve observers participated in three image quality research contexts in which they were asked to freely and spontaneously describe the quality of the presented test images. Standard viewing conditions were used. The data shows which attributes are most relevant for each test context, and how they differentiate between the selected image contents and processing systems. The role of non-HVS based image quality analysis is discussed. Keywords: Image quality, cameras, subjective testing, IBQ, image processing pipes 1. INTRODUCTION Designers and manufacturers of imaging electronics need optimal test methods for tuning the performance of cameras, imaging components, and systems. When the image quality is rather low such as in early home television and even first mobile phone cameras, it is sufficient to use rather crude test charts and multifunctional test images in order to reveal relevant spatial, temporal and color problems in the image. Comparison of the original and displayed image is a simple way to guarantee fidelity of coding, transmission and display. However, often there is no reference image available, as is the case in everyday photography. Furthermore, present devices are capable of producing very high quality images and there is a need to evaluate their performance accordingly. It is not a trivial task to obtain test information that would best guide the tuning of very high quality devices and their components. There is a need to know what is good, excellent, acceptable, and even a pleasing in the style of an image. From the engineering point of view, it is also important to know why certain high-level subjective properties occur in a processed or a displayed image. This knowledge will be especially valuable in the near future when ultra-high image quality becomes as common a property as it already is in print. Currently, there are no automatic objective test methods available that would give this information and even the subjective image quality test systems are just maturing. In this paper we describe a general framework for subjective testing of high quality images and give some examples of our approach in subjective testing of cameras and camera components. The benefits and challenges of this approach are discussed. The quality of an image produced by a digital still or video camera is a product of two multivariate quality pools that are dependent on each other: one technical and the other subjective. The technical quality pool consists of the camera, its electronic and optical components, and the imaging conditions. The subjective quality pool consists of the detected and experienced, subjective factors related to image quality. Examples of these subjective factors or attributes are naturalness, lightness, colorfulness, clarity, and appeal that together contribute to the experience of image quality. There are well-known measurement frameworks 1,2 that suggest how image quality should, in general, be measured in subjective image quality evaluations, but typically they do not suggest how to simultaneously measure subjective attributes in different image contexts. As surprising as it may seem, it is not known, how these different subjective image quality aspects together contribute to the overall image quality evaluation although candidate models have been suggested 2 . We have earlier developed a subjective measurement system that is aimed at finding out those subjective Image Quality and System Performance V, edited by Susan P. Farnand, Frans Gaykema, Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6808, 68080N, © 2008 SPIE-IS&T · 0277-786X/08/$18 SPIE-IS&T Vol. 6808 68080N-1 2008 SPIE Digital Library -- Subscriber Archive Copy