XXIX SIMP ´ OSIO BRASILEIRO DE TELECOMUNICAC ¸ ˜ OES - SBrT’11, 02-05 DE OUTUBRO DE 2011, CURITIBA, PR Monitoring Scenario for Non-Intrusive Quality Assessment of Teleconference Systems Leonardo O. Nunes, Fl´ avio R. ´ Avila, Luiz W. P. Biscainho, Bowon Lee, Amir Said, Ronald W. Schafer Resumo— A demanda por alta qualidade de experiˆ encia em sistemas de comunicac ¸˜ ao tem aumentado com a utilizac ¸˜ ao de sistemas de teleconferˆ encia, o que, por sua vez, estimulou a procura por sistemas confi´ aveis de avaliac ¸˜ ao de qualidade. Ferramentas para monitorac ¸˜ ao durante a operac ¸˜ ao do sistema, em particular, permitem que os sinais sendo transmitidos tenham a sua qualidade avaliada e as degradac ¸˜ oes que o est˜ ao atingindo identificadas sem interrupc ¸˜ ao do servic ¸o. Neste artigo, s˜ ao apre- sentados dois cen´ arios para monitorac ¸˜ ao de qualidade de sistemas de teleconferˆ encia, cada um contendo dois pontos de medic ¸˜ ao onde podem ser posicionadas ferramentas de monitorac ¸˜ ao n˜ ao- intrusiva (INMDs). S˜ ao levantados os tipos de degradac ¸˜ ao que podem ser monitorados em cada ponto de medic ¸˜ ao, al´ em de crit´ erios para o projeto dos respectivos INMDs. Os modelos pro- postos podem servir de guia para a gerac ¸˜ ao de sinais degradados adequados para testes subjetivos. Palavras-Chave— teleconferˆ encia, degradac ¸˜ oes ac´ usticas, avaliac ¸˜ ao n˜ ao-intrusiva de qualidade, INMD Abstract— Teleconference systems have increased the demand for high quality of experience in speech communication, which, in turn, calls for reliable quality assessment tools. In-service monitoring tools, in particular, aim to assess the quality of the transmitted signal as well as to identify impairments that might occur during the operation of the system. This paper presents two scenarios for quality monitoring of teleconference systems, each of them containing two measurement points where in-service non-intrusive measurement devices (INMDs) can be located. The impairment types that can be monitored at each measurement point are elicited, as well as design constraints on their respective INMDs. The proposed models can also guide the generation of impaired signals to be employed in subjective listening tests. Keywords— teleconference, acoustic degradations, non- intrusive quality assessment, INMD I. I NTRODUCTION The ubiquitous presence of speech communications has led to an increase in demand for quality of experience [14]. Mod- ern teleconference systems, in particular, call for dedicated quality assessment tools that guarantee the satisfaction of their users’ high expectation. The most reliable method for measuring the quality of a speech communication system is through subjective listening tests. In such tests, the quality of a given signal is measured by asking a group of subjects to grade it in a given scale, and averaging the attributed grades so as to obtain a single mean opinion score (MOS). Since the set-up and execution Leonardo O. Nunes, Flavio R. ´ Avila, and Luiz W. P. Biscainho are with the Signal Processing Laboratory, Federal University of Rio de Janeiro, Rio de Janeiro, Brazil. E-mail: ({lonnes, flavio, wagner}@lps.ufrj.br) Bowon Lee, Amir Said, and Ronald W. Schafer are with the Mobile and Immersive Experience Lab in HP Laboratories, Palo Alto, CA, USA. E-mail: ({bowon.lee, amir said, ron.schafer}@hp.com) of subjective listening tests is time-consuming and expen- sive [3], [8], [2], computational (so-called objective) methods to emulate them gained attention of both research and industry communities [13], [6]. These automatic quality assessment (QA) methods can be signal-based [14], such as the PESQ algorithm [13], or parametric [14], such as the E-Model [12]. Another possible way to classify these tools is regarding the stage in which they are to be employed in a communication system. A QA tool can be used during system design, mainte- nance, or operation 1 , each phase requiring different tools with varying specifications. During the design and maintenance phases, the operator has complete control over the system and can perform as many tests on the system as desired. Intrusive double-ended (i.e., where both degraded and non-degraded signals are avail- able) [14] QA tools can be employed to predict the perceptual quality of the transmission systems [5], [6], [18] and/or to identify possible problems [16]. A set of test points [2] must be properly defined, as briefly discussed in Section II-A, for this task. Quality assessment of systems during their operation, on the other hand, calls for tools that are able to work non- intrusively [14]. Such monitoring devices [17] must access the transmission system internally, in order to gather the necessary information without interruption. In ITU standard P.651 [11], different In-Service Non- Intrusive Measurement Devices (INMD) for voice-grade pa- rameters, in both circuit-switched and packed-based telephony networks, are specified. As defined in that document, “The parameters which are accessible via INMD measurements can only be obtained from the signals (of one direction or of both directions) at one specific point in the network.”, hence each INMD is located at a single point at the network and can be double-ended or single-ended (i.e., where only the degraded signal is available to the INMD) [14]. This paper describes a general model for teleconference systems including measurement points where INMDs can be connected, and elicits degradations that can be identi- fied/assessed at each location. Ideally, an INMD located at a given measurement point could house a signal-based [15] QA tool or a degradation type classifier [16] responsible for monitoring the quality of the telepresence system during its operation. The structure of the paper is as follows. In Section II, the teleconference model originally presented in [2] is briefly described. The new non-intrusive scenario for teleconference 1 A similar categorization is performed in [15].