1 All authors contributed equally. 644 2003 — Ninth Americas Conference on Information Systems TEST PERFORMANCE AND THE MEDIUM: UNEARTHING DIFFERENCES THAT MAKE A DIFFERENCE Jennifer Nicholson 1 Washington State University jnicholson@wsu.edu Andrew Hardin Washington State University ahardin@wsu.edu Darren Nicholson Washington State University dbnicholson@wsu.edu Abstract This study will attempt to provide an explanation for the disparity that exists between the literature that purports no difference between individuals’ performance when taking paper-and-pencil versus computer-based tests, with the literature that purports that high levels of computer anxiety will influence computer-based performance. The question is asked, how can the findings that show high levels of computer anxiety negatively influence performance co-exist with findings that show that there is no difference between an individual’s performance when taking a paper-and-pencil versus a computer-based exam? This research posits that although there may be no difference in test performance between pencil-and-paper and computer-based tests for simple questions, there will be a test performance difference for questions that are complex. Thus, we also ask what is the emergent phenomenon that is occurring that causes computer anxiety to reduce performance on complex questions administered via the computer? To explain this phenomenon, we will draw upon the cognitive psychology literature, specifically the literature regarding cognitive capacity, to show that some of an individual’s cognitive capacity will be utilized by emotional arousal (e.g. computer anxiety), thereby increasing the amount of cognitive load on an individual’s working memory. The increase in cognitive load on working memory, namely computer anxiety, should not influence an individual’s computer-based test performance for simple questions, however, it is posited that it will influence an individual’s computer-based test performance for questions that are complex. Introduction With the pervasiveness of computers has come the increased use of computerized testing in both educational and organizational settings. Although paper-and-pencil testing will never become extinct, computerized testing is becoming the norm in both Internet-based distance education courses and traditional courses supplemented with web-based materials and exercises (Klass and Crothers, 2000). With this inevitable shift in test administration mode, it has thus become critical to examine the differences, if any, that computerized testing may have on an individual’s test-taking performance. It would be foolish to operate under the assumption that as long as identical tests are given, it does not matter what administration mode is used. It is important at the outset to note the different terms used to describe tests that are administered via computer. Computerized testing is the most general term and encompasses any type of testing using a computer. Computer-based testing involves using a computer to administer the same test as that given in a paper-and-pencil format; whereas computer-adaptive testing administers test questions based on the examinee’s answer to the previous question or questions. This study is primarily concerned with