Test administrator’s gender affects female and male students’ self-estimated verbal general knowledge Tuulia M. Ortner * , Isabella Vormittag Division for Psychological Assessment, Department of Psychology, Free University Berlin, Habelschwerdter Allee 45, D-14195 Berlin, Germany Received 19 March 2009; revised 22 July 2009; accepted 29 September 2009 Abstract Effects of test administrator’s gender on test takers’ self-estimated verbal general knowledge and de facto verbal general knowledge were investigated. Based on three theories previously applied in research dealing with the effects of test administrator’s ethnicity, it was expected male and female test takers to show higher scores under female test administration. In a double-blind face-to-face-testing design, 93 university students of both genders in four groups were tested by 20 test administrators of both genders. A MANOVA confirmed the expected significant main effect. Female and male students reached higher scores in self-estimated knowledge when tested by a female test administrator in comparison to female students and male students tested by a male test administrator (Cohen’s d ¼ 0.46). No significant effects resulted for de facto knowledge. Ó 2009 Elsevier Ltd. All rights reserved. Keywords: Test administration effects; Gender stereotype; Stereotype threat; Self-estimation; Metacognition 1. Introduction Performing a cognitive task during a learning process, on an exam, or in a standardized test situation is a dynamic process that is always situated in a particular context. Acquired knowledge does not necessarily lead to a successful performance (Schutz & Davis, 2000). Within the field of educational and employment testing, efforts are being made to reduce assessment bias by targeting effects that are believed to systematically impair test performance. A fair assessment procedure is supposed to provide comparable opportunities for examinees to demonstrate acquired knowledge and skills that are relevant to the test’s purpose (Willingham & Cole, 1997). However, this requirement is often not given in testing prac- tice; even within standardized testing procedures, ethnic, socioeconomic, or gender characteristics might have an impact on performance. In recent times, several mechanisms and testing conditions have been identified as leading to systematic differences in test resultsda form of bias (Marx & Stapel, 2005; Wheeler & Petty, 2001). Related to this issue is the objectivity of an assessment procedure, that is, the indepen- dence of test results from the testing situation and the test administrator; this also must be considered as a possible source of systematic differences in test results. In past discussions, surprisingly little attention has been paid to the effects generated by the person who administers a testing procedure, especially with reference to gender. The test administrator, including his or her gender, cannot be purged: There is always at least one person who is responsible for introducing the test, for informing and securing consent in testing, as well as for intervening in cases of potentially bias-evoking conditions (e.g., Fernandez-Ballesteros et al., 2001). This person is a situational characteristic of the test, and could possibly evoke bias. Hence, we need to know more about systematic effects related to test administrator’s char- acteristics. The following manuscript addresses effects of test administrator’s gender on the self-estimation and performance of men and women when taking a test of general knowledge. * Corresponding author. Tel.: þ49 30 83855649; fax: þ49 30 83855647. E-mail addresses: tuulia.ortner@fu-berlin.de (T.M. Ortner), isabella. vomittag@fu-berlin.de (I. Vormittag). 0959-4752/$ - see front matter Ó 2009 Elsevier Ltd. All rights reserved. doi:10.1016/j.learninstruc.2009.09.003 Learning and Instruction 21 (2011) 14e21 www.elsevier.com/locate/learninstruc