c o m p u t e r m e t h o d s a n d p r o g r a m s i n b i o m e d i c i n e 1 1 3 ( 2 0 1 4 ) 383–395 jo ur nal ho me p ag e: www.intl.elsevierhealt h.com/journals/cmpb A usability evaluation of medical software at an expert conference setting Raymond Robert Bond * , Dewar D. Finlay, Chris D. Nugent, George Moore, Daniel Guldenring University of Ulster, Northern Ireland, UK a r t i c l e i n f o Article history: Received 4 December 2012 Received in revised form 2 October 2013 Accepted 5 October 2013 Keywords: Usability Interfaces ECG Medical software Interaction design HCI a b s t r a c t Introduction: A usability test was employed to evaluate two medical software applications at an expert conference setting. One software application is a medical diagnostic tool (electro- cardiogram [ECG] viewer) and the other is a medical research tool (electrode misplacement simulator [EMS]). These novel applications have yet to be adopted by the healthcare domain, thus, (1) we wanted to determine the potential user acceptance of these applications and (2) we wanted to determine the feasibility of evaluating medical diagnostic and medical research software at a conference setting as opposed to the conventional laboratory setting. Methods: The medical diagnostic tool (ECG viewer) was evaluated using seven delegates and the medical research tool (EMS) was evaluated using 17 delegates that were recruited at the 2010 International Conference on Computing in Cardiology. Each delegate/participant was required to use the software and undertake a set of predefined tasks during the session breaks at the conference. User interactions with the software were recorded using screen- recording software. The ‘think-aloud’ protocol was also used to elicit verbal feedback from the participants whilst they attempted the pre-defined tasks. Before and after each session, participants completed a pre-test and a post-test questionnaire respectively. Results: The average duration of a usability session at the conference was 34.69 min (SD = 10.28). However, taking into account that 10 min was dedicated to the pre-test and post- test questionnaires, the average time dedication to user interaction of the medical software was 24.69 min (SD = 10.28). Given we have shown that usability data can be collected at con- ferences, this paper details the advantages of conference-based usability studies over the laboratory-based approach. For example, given delegates gather at one geographical loca- tion, a conference-based usability evaluation facilitates recruitment of a convenient sample of international subject experts. This would otherwise be very expensive to arrange. A conference-based approach also allows for data to be collected over a few days as opposed to months by avoiding administration duties normally involved in laboratory based approach, e.g. mailing invitation letters as part of a recruitment campaign. Following analysis of the user video recordings, 41 (previously unknown) use errors were identified in the advanced ECG viewer and 29 were identified in the EMS application. All use errors were given a consensus severity rating from two independent usability experts. Out of a rating scale of 4 (where 1 = cosmetic and 4 = critical), the average severity rating for ∗ Corresponding author. Tel.: +44 7762768678. E-mail address: rb.bond@ulster.ac.uk (R.R. Bond). 0169-2607/$ – see front matter © 2013 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.cmpb.2013.10.006