Usability testing of a respiratory interface using computer screen and facial expressions videos Ana Oliveira a , Cátia Pinho a,c , Sandra Monteiro a , Ana Marcos a , Alda Marques a,b,n a School of Health Sciences, University of Aveiro (ESSUA), Campus Universitário de Santiago, Edifício 30, Agras do Crasto - Campus Universitário de Santiago, 3810-193 Aveiro, Portugal b Unidade de Investigação e Formação sobre Adultos e Idosos (UNIFAI), Largo Prof. Abel Salazar, 4099-003 Porto, Portugal c Institute of Electronics and Telematics Engineering of Aveiro (IEETA), University of Aveiro, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal article info Article history: Received 28 June 2013 Accepted 8 October 2013 Keywords: Graphical user interface Usability testing Facial videos Screen videos Observer XT abstract Computer screen videos (CSVs) and users' facial expressions videos (FEVs) are recommended to evaluate systems performance. However, software combining both methods is often non-accessible in clinical research fields. The Observer-XT software is commonly used for clinical research to assess human behaviours. Thus, this study reports on the combination of CSVs and FEVs, to evaluate a graphical user interface (GUI). Eight physiotherapists entered clinical information in the GUI while CSVs and FEVs were collected. The frequency and duration of a list of behaviours found in FEVs were analysed using the Observer-XT- 10.5. Simultaneously, the frequency and duration of usability problems of CSVs were manually registered. CSVs and FEVs timelines were also matched to verify combinations. The analysis of FEVs revealed that the category most frequently observed in users behaviour was the eye contact with the screen (ECS, 32 79) whilst verbal communication achieved the highest duration (14.8 76.9 min). Regarding the CSVs, 64 problems, related with the interface (73%) and the user (27%), were found. In total, 135 usability problems were identified by combining both methods. The majority were reported through verbal communication (45.8%) and ECS (40.8%). “False alarms” and “misses” did not cause quantifiable reactions and the facial expressions problems were mainly related with the lack of familiarity (55.4%) felt by users when interacting with the interface. These findings encourage the use of Observer-XT-10.5 to conduct small usability sessions, as it identifies emergent groups of problems by combining methods. However, to validate final versions of systems further validation should be conducted using specialized software. & 2013 Elsevier Ltd. All rights reserved. 1. Introduction Healthcare professionals are increasingly challenged to acquire and manage large amounts of information, while still providing high quality health services. Thus, healthcare informa- tion systems (HCISs) have become vital to store, organise and share clinical information, which facilitates and improves health professionals' decision making [1]. Although health professionals are the major beneficiaries of these technologies, they often resist to their implementation [2,3]. This resistance have been attrib- uted to the felling of loss of control expressed by health professionals when interacting with systems [4]. Furthermore, computer systems are often developed by professionals outside the health field who often do not have a full understanding of clinical evaluations and procedures [5]. This may affect the construction of the system by being complex and difficult to navigate, contributing to health professionals' resistance to its use. Therefore, systems evaluations performed with the end users are essential, not only in the final version, but throughout the progress cycle to guarantee that the system is developed accord- ing to health professionals standards, ensuring its effectiveness, efficiency and usability [6,7]. To verify and optimise systems, analytical and empirical methods from the area of usability engineering and human–computer-interaction have been applied in HCISs evaluation studies [8]. Kushniruk and Patel [5,9] have been researching in the field of usability testing and proposed different types of data collection, such as video recordings of the computer screens and users while performing tasks and think- aloud reports. Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/cbm Computers in Biology and Medicine 0010-4825/$ - see front matter & 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.compbiomed.2013.10.010 n Corresponding author at: School of Health Sciences, University of Aveiro (ESSUA), Edifício 30, Agras do Crasto - Campus Universitário de Santiago, 3810- 193 Aveiro, Portugal. Tel.: þ351 234372462; fax: þ351 234401597. E-mail addresses: alao@ua.pt (A. Oliveira), catiap@ua.pt (C. Pinho), sandramonteiro@ua.pt (S. Monteiro), anajorgesm@ua.pt (A. Marcos). amarques@ua.pt (A. Marques). Computers in Biology and Medicine 43 (2013) 2205–2213