Assessment of the Effects of User Characteristics on Mental Models of Information Retrieval Systems Xiangmin Zhang Library and Information Science Program, Wayne State University, Detroit, MI 48202., E-mail: ae9101@wayne.edu Mark Chignell Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Canada. E-mail: chignel@mie.utoronto.ca This article reports the results of a study that investi- gated effects of four user characteristics on users’ men- tal models of information retrieval systems: educational and professional status, first language, academic back- ground, and computer experience. The repertory grid technique was used in the study. Using this method, important components of information retrieval systems were represented by nine concepts, based on four IR experts’ judgments. Users’ mental models were repre- sented by factor scores that were derived from users’ matrices of concept ratings on different attributes of the concepts. The study found that educational and profes- sional status, academic background, and computer ex- perience had significant effects in differentiating users on their factor scores. First language had a borderline effect, but the effect was not significant enough at   0.05 level. Specific different views regarding IR sys- tems among different groups of users are described and discussed. Implications of the study for information sci- ence and IR system designs are suggested. Introduction User diversity calls for information retrieval (IR) sys- tems that can accommodate heterogeneous user groups (Allen, 1996). To design such systems, it is necessary to identify and understand different types of users. Efforts on user-centered research in IR have been made for decades. Individual difference studies in IR have found that user search performance varied on certain user charac- teristics (Bellardo, 1985; Borgman, 1989; Charoenkitkarn, 1996; Fenichel, 1981; Kamala, 1991; Marchionini et al., 1993; Qiu, 1993; Woelfl, 1984; Yee, 1993). Such charac- teristics include a user’s experience with a system, academic background, age, gender, and personality (Borgman, 1989; Egan, 1988). For example, in terms of academic back- ground, Borgman (1984b) and Kamala (1991) found sci- ence/engineering majors had better search performance than social sciences/humanities majors did. In general, this type of studies found that search perfor- mance differences exist on certain user characteristics and the findings are helpful for understanding users, to some extent. The major limitation of these studies, however, is that the cognitive processes or the reasons why one type of users would perform better than or differently from another type of users were not studied. Further investigations are needed to better understand the performance differences as well as how to effectively design IR systems that accom- modate these differences (Savage-Knepshield & Belkin, 1999). For system designs, we not only need to know on what characteristics users’ behavior or search performance would vary, but also need to know why: Why different types of users behave in different ways and have different search performance. Research in IR interaction emphasizes searching as an interactive task and the user’s interaction with IR systems. In addition to the comparisons between different system/ interface designs for revealing the nature of interactions and system/user search performance (Voorhees & Harman, 1998), a number of interactive IR models have been pro- posed, such as Belkin’s (Belkin & Vickery,1985; Belkin et al., 1995) episode model, Ingwersen’s (1992) cognitive model, and Saracevic’s (Saracevic, 1997; Spink & Saracevic, 1997) stratified model. These models not only reflect the interactive nature of IR systems but also try to explain, from different perspectives, at the cognitive level the reasoning processes behind user’s interaction with IR systems. While providing frameworks for studying users in the process of directly consulting an IR system (Robins, 2000), Received February 22, 1999; Revised July 6, 2000; accepted July 25, 2000. © 2001 John Wiley & Sons, Inc. ● Published online 15 February 2001 JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGY, 52(6):445– 459, 2001