Comparison of Testing Environments with Children for Usability Problem Identification Mohammadi Akheela Khanum #1 , Munesh Chandra Trivedi *2 # PAHER University Udaipur, India 1 akheela.khanum@gmail.com * Dehradun Institute of Technology Greater Noida, India 2 Munesh.trivedi@gmail.com Abstract— This paper report the results of an empirical study which was carried out to investigate the effect of testing environment on the results of usability evaluation process. The study involved 54 school children from India in the age range of 11-13 years. Children were asked to perform books searching tasks with International Children’s Digital Library (ICDL). Children’s activities with ICDL were captured by using CamStudio, an open source screen capturing software. The effect was quantified in terms of number of usability problems identified by the children when they were tested in various testing environments. The results are indicative that field testing with children can be a viable solution in terms of reduced time taken to complete the given tasks and reduced frustration levels reported by the children during the tests. Keywords- usability evaluation; children; think-aloud; constructive interaction; problem identification; I. INTRODUCTION Children of today, born after the emergence of the Internet, are considered millennial [1] that have been born digital and raised as “Digital Natives” [2][3]. The design and evaluation of children’s technologies have received increased attention during the last several years [4]. Children should be considered individuals with strong opinions, needs, likes, and dislikes, and they should be treated as such [5]. When evaluating technologies with children, evaluators are typically faced with unique challenges as children enter usability evaluation with special preconditions [6]. Thus, we need to understand how to create successful environments for children that facilitates usability problem identification. Studies have shown that children are mostly affected by the context than adults. Context carries various meanings, in this study we refer to context as the physical location. The choice of location as context during usability evaluation is considered an important topic of discussion in research. Typically, the choice is between evaluating in an artificial setting such as a laboratory or in a more natural setting through a field evaluation. Children show varying behavior when they are tested in the laboratory environment and when they are tested in the field environment. The importance of the physical context has been explored and studied by several usability researchers. However, we still lack clear empirical evidence of the merits of one environment over the other during usability evaluation with children. The purpose of our study is to find the answers to the following research questions: RQ1: Are same usability problems found in both lab and field? RQ2: Is the severity of the problems same in lab and field? RQ3: Does the test environment affect the user performance? RQ4: Which setting is more suitable to test with the children? II. LITERATURE REVIEW The importance of physical context in usability evaluation has been researched for a long. Out of the many factors that can effect usability evaluation, physical context is considered to directly influence the behaviour of the people involved in the usability evaluation. The physical context may include the location, the temperature, the time, the light etc. Some of the popular researches in this area are as follows. Tullis et al. [7] compared remote and lab settings based on the time taken to complete the tasks and the problems discovered. Their study involved a prototype of a Web site for providing the employees of a company with access to information about their own benefits, including retirement savings information, pension information, medical and dental coverage, payroll deductions and direct deposit, and financial planning. The results found no significant difference between remote and traditional task times. Both remote and traditional lab testing revealed usability issues on existing websites. However, Tullis’s participants scored the subjective tasks and interface differently between the different testing locations. It was concluded that the remote condition would incite Mohammadi Akheela Khanum et.al / International Journal of Engineering and Technology (IJET) ISSN : 0975-4024 Vol 5 No 3 Jun-Jul 2013 2048