Evaluating the Usability of Interactive Digital Television Applications Andrés Solano, César Collazos Departamento de Sistemas, FIET Universidad del Cauca Popayán, Colombia {afsolano, ccollazo@unicauca.edu.co Cristian Rusu Escuela de Ingeniería Informática Pontificia Universidad Católica de Valparaíso Valparaíso, Chile cristian.rusu@ucv.cl Luis Merchán Facultad de Ingeniería Universidad San Buenaventura Cali, Colombia lmerchan@usbcali.edu.co Abstract—The evaluation of the usability of an interactive system is one of the most important steps in the user-centered design approach. Usability evaluation methods are quite diverse and many of them may be used to evaluate Interactive Digital Television (iTV) applications. Questions arise when deciding which method and/or combination of methods give more (relevant) information. Three iTV applications have been evaluated, applying several usability evaluation methods. Based on the effectiveness and efficiency of these methods, this paper proposes a methodology to evaluate the usability of iTV applications. Keywords-usability; usability evaluation methods; interactive digital television; applications; methodology. I. INTRODUCTION Usability evaluation of an interactive system is one of the most important stages in the user-centered design approach. It examines each characteristic of the system’s usability and judges the extent to which the attributes, paradigms, and usability principles have been applied [1]. Interactive Digital Television (iTV) is growing in popularity and it already occupies an important role in our society [2]. This makes it a medium with great potential for users to access interactive applications that can entertain, inform, communicate, etc. As iTV applications become more widely-available to the general public, users are less experienced in handling them. Consequently, the usability of such applications becomes more relevant every day. Usability is a quality attribute of software which includes a set of metrics and methods in order to obtain easy to use and learn systems [3]. The usability evaluation methods (UEMs) are quite diverse. Many of them can be applied in order to evaluate iTV applications. Their use depends on variables such as cost, availability of time, human resources. However, the choice of the appropriate evaluation method(s) is not simple. Each UEM has strengths and weaknesses, and each one focuses on the evaluation of specific aspects of usability. Furthermore, UEM usually complement each other [4][5]. There is a need for a specific usability evaluation methodology for iTV applications. Thus, this work focuses on the study of the UEMs which apply to iTV applications. It aims to identify a set of methods that may lead to a methodology in order to provide more complete usability information than the indiscriminate and independent application of evaluation methods. Hence, this article analyzes the results of the usability evaluations of three iTV applications, through several methods. Section 2 presents the key theoretical concepts and section 3 presents the related works. Section 4 presents the usability evaluation experiments. Subsequently, the analysis of the results obtained in applying the UEMs is presented in section 5. Section 6 presents a methodology to evaluate the usability of iTV applications. Section 7 presents preliminary conclusions and future works. II. THEORETICAL CONCEPTS A. Usability The ISO/IEC 9241 defines usability as "the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use" [6]. Usability evaluation is the activity that applies a set of methods for analyzing the quality of use of an interactive system in different stages of the development life cycle [7]. The evaluation process can be implemented by people with different skills and knowledge, such as usability experts, current and potential users, or system’s designers. Usability evaluation is necessary in order to validate the usability requirements of the product [8]. B. Usability evaluation methods There are several UEMs that can be carried out on different representations of the system: early (paper) prototypes, functional prototypes, and final prototypes/systems. Most of the methods facilitate the communication between the user and the usability evaluator, since the latter has to identify the aims, perceptions, problems, and questions of users [9]. The UEMs are usually classified into two groups [9]: