COMPARISON OF HEURISTIC EVALUATION AND THINK ALOUD METHODS: A STUDY IN RADIOTHERAPY CONTOURING SOFTWARE Anjana Ramkumar 1 , Yu Song 1 , Edit Varga 1 , Wiro J. Niessen 1,2 , Anne Laprie 3 , Ben Rowland 3 , Adinda Freudenthal 1 † 1-Delft University of Technology, 2-Erasmus MC - University Medical Center, 3-Institut Claudius Regaud During the radiotherapy planning process patient’s medical images are used to determine the optimum configuration of radiation beams for their cancer treatment. The primary aim of radiotherapy planning is to maximize radiation dose to the patient’s tumour while sparing their normal tissues. To achieve this, the boundary of the tumour needs to be accurately identified in the treatment planning process, especially in the step of target volume delineation. Clinically, tumour delineation is performed by physicians, either manually or using semi-automatic/automatic software based on the patient’s CT, PET and/or MRI images. In addition to the expertise of the physician and choices of segmentation algorithms, the usability of the software also plays an important role in the tumour delineation process. In this paper, the usability of the IPLAN ® contouring software was evaluated. First, two evaluators assessed the software according to the heuristic evaluation method. Then three physicians evaluated the same system using the think aloud method. The outcomes of the experiments revealed different insights of the system. With the think aloud method, it was easier to identify end users’ preferences regarding both software interface and hardware input tools. On the other hand, the heuristic evaluation method uncovered more specific issues with the interface. Besides, this method was able to explore more details regarding individual functionalities on the interface. Based on the comparison of the outcomes of both methods, it is suggested that in the process of improving usability of the contouring interface, the think aloud method can be applied to explore preferences of the user. The heuristic evaluation can be applied in designing the details of the interface. 1. INTRODUCTION The primary aim of radiotherapy planning is to maximize the delivery of radiation dose to the tumour while sparing normal tissues. In general, the process of radiotherapy planning can be divided into 8 steps: diagnosis, multi- disciplinary meeting, external radiotherapy patient consultancy, planning preparation, image fusion, contouring, dose prescription and dosimetry (Aselmaa et al., 2013). Each of these steps is a potential source of errors in the planning. For instance, possible errors may be made in setting up of the patient, determining the movements of the patient, identifying the position of the tumour and delineation (contouring) of the target volume etc. (Fraass, 2008) Those errors may cause insufficient radiation dose coverage of the targeted tumour and/or an overdosage of normal tissues. Of the 8 radiotherapy planning steps, tumour delineation contributes the most significant errors (Njeh et al., 2008). Technically, tumour delineation is the process of outlining the tumour boundary by manual, semi-automatic and/or automatic methods. In past decades, many types of software have been developed to aid this process (Brændengen et al, 2011). However, clinical use of these software systems reveals the following problems: 1. Most available interface are rather complicated and in many cases, all options are presented without clear indications (groups) of the importance (Chan et al., 2005); 2. A lot of inefficient Human Computer Interaction (HCI) is needed in the use of the systems. For instance, it takes a lot of time for physicians to contour the tumour precisely as they have to compare different datasets from CT (Computed Tomography), PET(Positron Emission Tomography) and MRI (Magnetic Resonance Imaging), and contour on a stack of 2D ( 2 Dimensional) images slice by slice manually using mouse and/or pen-tablet input devices (Ramkumar et al., 2013) 3. Rich medical knowledge is often needed in the process. As manual contouring is still the preferred method of many systems (Velazquez et al, 2012), physicians get little help in the contouring process, regarding difficult areas in medical images. This is especially true in tumour delineations due to the pathological uniqueness of each tumour. In addition to the limited abilities of different types of inputs of numerical algorithms, human factors is another source of error. Generally, human factor concerns interactions among people, system and their environment to improve the system performance (Hossain, 2014). The human factors involved in the tumour delineation include the physicians and their background, their cognitive abilities, their emotion, their physiological parameters, the context of physicians in the usage scenarios; designed Human Computer Interaction (HCI); the moment and time span interactions, and interfaces that enable those interactions. In the designing of a tumour delineation software for physicians, human factors should be considered before implementation. Experience has shown that costs and time taken for making changes later can be reduced if issues are identified in the early stage of the design (Shen et al, 2013). For this reason, studying current software solutions and practices of physicians are essential at the beginning of the design process. Among the human factors in the usage of radiotherapy software, the usability of interfaces is an important issue. †Deceased 18 February 2014