Quality of Experience Measurements for Video Streaming over Wireless Networks Kandaraj Piamrat ∗ , C´ esar Viho † , Adlen Ksentini † and Jean-Marie Bonnin ‡ ∗ INRIA Rennes-Bretagne Atlantique, Campus Beaulieu 35042 Rennes, France Telephone: (33) 299847137, Fax: (33) 299847171, Email: kandaraj.piamrat@inria.fr † IRISA/University of Rennes I, Campus Beaulieu 35042 Rennes, France Telephone: (33) 299847416, (33) 299847142, Fax: (33) 299847171, Email: {adlen.ksentini, cesar.viho}@irisa.fr ‡ Telecom Bretagne 2, rue de la Chˆ ataigneraie CS-17607 35576 Cesson-S´ evign´ e, France Telephone: (33) 299127007, Fax: (33) 299127030, Email: jm.bonnin@telecom-bretagne.eu Abstract—As wireless networks have been increasingly deployed, the need of quality measurement became essential since network operators want to control their network resources while maintaining user satisfaction. More importantly, measurement of technical parameters fails to give an account of the user experience, what could be named QoE (Quality of Experience). Therefore, many techniques have been developed in order to assess as accurately as possible this perceptual quality. To investigate QoE measurement, this paper presents three approaches namely subjective approach, objective approach, and hybrid approach. It also presents performance evaluation of these approaches for assessing QoE in video streaming application over wireless networks in different network conditions (using variation of loss rate and its distribution). We focus more specifically on a hybrid approach called Pseudo Subjective Quality Assessment (PSQA) that keeps advantages of both subjective and objective schemes while minimizing their drawbacks. We demonstrate that this approach provides good estimations comparing to the well-known objective metric called Peak Signal to Noise Ratio (PSNR). We also observe that PSQA gives similar result comparing to subjective test that has been evaluated by human observers in most of the cases. Moreover, one objective of this evaluation is to validate PSQA for QoE measurement, which will facilitate the use of QoE as metric for resource management in the future. For that, we also give some possible directions allowing us to manage network resources using this metric. Index Terms—Quality of Experience (QoE), QoE Assessment, Video Streaming, Wireless Networks, Resource Management I. I NTRODUCTION Before the beginning of multimedia communication era, simple parameters such as bandwidth, loss, delay or other network related parameters were enough to evaluate quality of service because the provided services are plain applications such as e-mail or file transfer. These applications do not have complicated definition of quality, for example, with file transfer, bandwidth or delay would probably be sufficient to imply quality of service. However, nowadays real-time multimedia applications are being deployed on IP networks and technical parameters can no longer assess accurately the quality of service as it is perceived by human. Users expect to have good perceptual quality that can be derived from many factors including not only technical parameters but also users experience. Network operators need to control their resource while maintaining user satisfaction, which will result in user fidelity and benefit for the company. Therefore, they need to take into account not only quality of service (QoS) but also quality of experience (QoE). Quality of experience [1] is the overall acceptability of an application or service, as perceived subjectively by end-users. It is basically a subjective measurement of end-to-end performance at the service level, from the point of view of the users. In the following, we give an overview of QoE assessment in three different ways: subjective, objective, and hybrid respectively. 1) Subjective Assessment The most accurate approach to assess perceived quality is the subjective assessment because there is no better indicator of video quality than the one given by humans. However, the quality score given by a human also depends on his/her own experience. The assessment consists in building a panel of human observers which will evaluate sequences of video depending on their point of view and their perception. The output of the test is in terms of mean opinion score (MOS) [2], which is given in Table I below. TABLE I MEAN OPINION SCORE - MOS MOS Quality Impairment 5 Excellent Imperceptible 4 Good Perceptible but not annoying 3 Fair Slightly annoying 2 Poor Annoying 1 Bad Very annoying Some standard methods for conducting subjective video quality evaluations exist such as the ITU-R BT.500-11 [3]. Its variations are Single Stimulus (SS), Double Stimulus Im- pairment Scale (DSIS), Double Stimulus Continuous Quality Scale (DSCQS), Single Stimulus Continuous Quality Evalua- tion (SSCQE), Simultaneous Double Stimulus for Continuous Evaluation (SDSCE), and Stimulus Comparison Adjectival Categorical Judgment (SCACJ). All the variations are pretty much similar; changes concern for example, evaluation scale, 2009 Sixth International Conference on Information Technology: New Generations 978-0-7695-3596-8/09 $25.00 © 2009 IEEE DOI 10.1109/ITNG.2009.121 1184 2009 Sixth International Conference on Information Technology: New Generations 978-0-7695-3596-8/09 $25.00 © 2009 IEEE DOI 10.1109/ITNG.2009.121 1184 Authorized licensed use limited to: Claudio De Castro Monteiro. Downloaded on December 17, 2009 at 15:07 from IEEE Xplore. Restrictions apply.