PERCEPTUAL QUALITY EVALUATION ON PERIODIC FRAME-DROPPING VIDEO Zhongkang Lu†, Weis Lin‡, Boon Choong Seng§, Sadaatsu Kato§, Eeping Ong†and Susu Yao† †Institute for Infocomm Research, Agency for Science, Technology and Research, 21 Heng Mui Keng Terrace, Singapore 119613; ‡School of Computer Engineering, Nanyang Technological University, Nanyang Avenue, Singapore 639798. §Research Laboratories, NTT DoCoMo, Inc., Kanagawa, 239-8536 Japan. ABSTRACT In general, low and very-low bitrate video communication systems cannot achieve the full frame rate (25Hz for PAL or 30Hz for NSTC), which brings temporal distortion. The in- fluence of reduced frame rate on subjective quality evaluation is a very important topic for both perceptual quality metrics and communication system optimization. In this paper, we present our work on the numerical modeling of the influence. The work includes two parts: first we measured the detectabil- ity and annoyance of periodic frame dropping’s effect on per- ceptual visual quality evaluation under different content and frame size conditions. Then, a simple and effective feature is proposed to represent the content of video in temporal qual- ity evaluation. The high Pearson and Spearman correlation results between the MOS and proposed model, as well as the results of other two error metrics, confirm the success of the selected temporal content-quality feature. Index Terms— Visual Quality, Periodic Frame-dropping, Temporal Content-Quality Feature 1. INTRODUCTION The distortion of a very-low bitrate compressed video gener- ally includes two parts: spatial distortion and temporal dis- tortion. Considering the spatial and temporal information are processed in different cortex in human brain, the subjective evaluation of the distortion can be modeled by a nonlinear combination of spatial distortion and temporal distortion. Up to now, most of current video quality metrics only consider spatial distortion. Moreover, because the temporal distortion in very-low bitrate compressed video belongs to suprathresh- old distortion condition [1], temporal CSF doesn’t work well in the application [2] [3]. This work is to isolate the sub- jective evaluation of temporal distortion from spatial distor- tion, and to establish a model to accurately reflect the rela- tionship among subjective evaluation of temporal distortion, frame rate and content of video. Much research showed that the perception of continuous motion is very complex. It concerns a manifestation of com- plex functions. For example, the perception of first-order (luminance-defined) motion and second-order (contrast-defined) motion are processed in different areas in the human brain, and the associated processing mechanisms are very differ- ent [5] [6]. Substantial psychological and physiological re- search also showed that the sensitivity of apparent motion perception can be analyzed by a number of spatiotemporal energy models [7], which suggests that the motion content in video plays an important role on temporal quality evaluation. This paper aims at the modeling of the influence of peri- odic frame-dropping on visual quality perception. The model is established under two conditions: 1. no-reference condition, which means the information of original sequences are not used; and 2. prediction condition, which means the input to the model is the motion representation of original sequences and the target frame rate. The proposed models not only can be used into the design of visual quality metrics (first condition), but also on rate- distortion optimization for very-low bitrate video communi- cation (second condition). The work includes two parts: first is a subjective experi- ment to measure the detectability and annoyance on periodic frame dropped video sequences, without the introduction of spatial distortions. Then, based on the subjective experiment results, a simple and effective feature is found to represent the content of video in temporal quality evaluation. The pa- per is organized as following: section 2 gives an introduction of the subjective experiment. The proposed temporal content- quality feature is presented in section 3. Experimental results and analysis are given in section 4. Section 5 is the conclu- sions and future works. 2. SUBJECTIVE VIEWING EXPERIMENT The subjective viewing experiment is to measure the detectabil- ity and annoyance of frame dropping’s effect on perceptual III - 433 1-4244-1437-7/07/$20.00 ©2007 IEEE ICIP 2007