Abstract Usability testing has become one of the key aspects of user centered design. It typically employs various lab setups equipped with cameras and other devices to monitor the user’s interactions with the software. Psychology studies on Electronic Performance Monitoring suggest that the more monitoring equipment used, the higher the anxiety levels of the person being observed. This study observed how usability labs employing various methods of user monitoring affected user anxiety levels and performance. If increased anxiety levels are causing users to make more errors, then the lab environment could be impacting the results of usability testing. The study examined three groups of subjects performing a usability test under different electronic and direct monitoring conditions. Each lab had significantly different layouts and employed different methods of monitoring. The labs were designed to be increasingly more intrusive in the monitoring set-ups. The study measured the anxiety levels of all participants after the test and counted the number of errors made. There were statistically significant differences, suggesting that lab environment can impact usability test results. Keywords usability testing, usability lab, lab design, user anxiety, lab anxiety, and user testing. I. Introduction One of the main reasons that software is often difficult to use is that the software developer has little or no contact with the people who will eventually be using the software. As a result, the developer’s focus tends to be entirely on the design of the underlying system, with consideration being given only to what will make the software easier to implement rather than easier to use [1]. It is with this concept that software designers incorporate the intended user into the design process. This is known as User Centered Design (UCD). UCD keeps the user as the focus of the design process using a variety of methods including interviews, job shadowing, and usability testing [1]. Usability testing is generally conducted in a lab specifically designed to observe a user’s actions and reactions to an application. Usability labs in industry utilize a variety of different layouts. Figures 1, 2, and 3 are all images of usability labs in the St. Louis area. Most usability labs contain at least one kind of monitoring device. A few of the more common examples are cameras, voice recorders, and two-way mirrors. As can be seen in Figures 1-3, not all labs use all of these monitoring methods. Some labs use small cameras mounted on the desk (Figure 3), while others use cameras mounted on tripods around the user (Figure 2). Figure 1 MasterCard Intl. Usability Lab Figure 2 Edward Jones Usability Lab Figure 3 Tripos Inc. Usability Lab However common the accepted methods of usability testing are, there still exist several questions about the way they are implemented. One consideration is the potential effect of Electronic Performance Monitoring (EPM). EPM is the use of electrical monitoring devices such as cameras, voice recorders, or eye movement trackers to capture or monitor the performance of a subject, usually an employee [2]. EPM is very common in usability testing as there is always the need to capture what the user is doing for review at a later time. According to psychological studies, EPM causes detrimental effects on workplace performance due to increased levels of anxiety [2]. Thus, the question arises THE EFFECTS OF THE TESTING ENVIRONMENT ON USER PERFORMANCE IN SOFTWARE USABILITY TESTING Bryan Grubaugh Dept. of Computer Science Southern Illinois University Edwardsville, IL 62026 USA bgrubau@siue.edu Dr. Susan Thomas Dept. of Psychology Southern Illinois University Edwardsville, IL 62026 USA suthoma@siue.edu Dr. Jerry Weinberg Dept. of Computer Science Southern Illinois University Edwardsville, IL 62026 USA jweinbe@siue.edu