Performance Analysis of a Video Quality Ruler Methodology for Subjective Quality Assessment Pedro Garcia Freitas a , Alexandre F. Silva a,b , Judith A. Redi c , Myl` ene C.Q. Farias a a University of Bras´ ılia, Bras´ ılia, Brazil b Federal Institute of Triˆ angulo Mineiro, Paracatu, Brazil. c Delft University of Technology, Delft, The Netherlands Abstract. Objective Video Quality Metrics are designed to be as reliable as the subjective quality assessments on which they are calibrated and validated. However, existing standard methodologies for subjective video quality assessment provide low reliable results for some conditions. In this study, we investigate whether an extension of the Quality Ruler experimental methodology, originally defined for images and shown to be more reliable than, e.g., standard Single Stimulus methods, can be adapted to reliably assess the quality of videos. The Video Quality Ruler methodology allows subjects to assess video quality using a set of reference anchor images (the ruler), spanning a wide range of quality altogether, but closely spaced in function of quality one from the other. Subjects are asked to compare the quality of the displayed test video with the quality of these anchor images, displayed on a tablet, and indicate which of the reference images matches in quality the test video. As a result, the video quality assessment task is reduced to a set of visual comparisons between video and reference image quality. In this paper, we describe how to adapt the original Quality Ruler methodology to video quality assessment, and we compare the proposed methodology with two other, widely used experimental methodologies: the Single Stimulus (SS) and the Double Stimulus (DS) method. Our results show that VQR is a reliable method to assess video quality according to a multitude of criteria. Keywords: Subjective quality assessment, Quality ruler, Subjective experiment, Testing conditions and methods.. *Pedro Garcia Freitas: sawp@sawp.com.br ; Myl` ene C.Q. Farias: mylene@ieee.org . 1 Introduction Videos are present in several services, applications, delivered over the Internet, and streamed through the most different types of devices. Cisco reports demonstrate that video content sur- pass 50% of total global traffic over the Internet and predict that 62% of the mobile traffic will be video until 2020. 1 Due to this growing demand, content broadcasters are always facing the chal- lenge of providing services that satisfy consumers expectations of quality. Therefore, the design metrics and methods that are able to assess the quality of video signals is a fundamental step to ensure a good experience for the end-users. Video quality assessment can be performed either by using computer algorithms (objective metrics) or by running psychophysical experiments (subjective testing). Subjective tests are con- 1