NordiCHI2000 Proceedings Stockholm October 23-25 2000  Copyright NordiCHI and STIMDI 2000. -1- Usability testing of interactive multimedia services Ebba Thora Hvannberg University of Iceland Hjardarhagi 2-6 IS-107 Reykjavik Iceland Telephone: (354) 525 4702 ebba@kerfi.hi.is Marta Kristin Larusdottir EJS ehf. Grensasvegi 10 IS-108 Reykjavik Iceland Telephone: (354) 563 3000 marta@ejs.is ABSTRACT Interactive multimedia services, such as video-on-demand offered to the public at home need to be tested carefully for usability. The environment as well as users is heterogeneous and they use different input and output devices than with a personal computer. User Interface designers need to learn more about the users, their environment and the hindrances they encounter. Two service trials were conducted where thirty families had access to video-on-demand, news-on-demand and worldwide web services for three months. The users had access to the services via a set-top-box connected to a television or via a personal computer. The paper describes how three methods: logging, thinking-aloud and questionnaires were applied as well as their results. Finally, the experiences from using the methods are discussed. Keywords Usability tests, interactive services, multimedia, home, high-speed network, thinking-aloud, access logs, interviews. 1 INTRODUCTION Two user trials of multimedia interactive services over a high-speed network to residential users were carried out in the AMUSE (Advanced Multimedia Services for Residential Users) project (AMUSE Final report, 1998). Ten homes participated in the first trial 1997 and twenty homes in the second trial 1998. In the first trial all users were equipped with a set-top-box that was connected to a TV and linked to a high-speed network. The user controls the set-top-box with a remote control and/or an infrared keyboard. In the second trial half the users had a set-top-box and the other half received a network card and a video playing card that was installed in their own personal computers. The clients were connected to a central video server via a high-speed network. Each trial lasted three months. The services that were offered were video-on-demand, news-on-demand and web browsing. Before, during and after the trials usability tests were carried out. In this paper we will describe the methodology that was used, our experience from the tests and the results. The