Assessing the Belief Bias Effect With ROCs: It’s a Response Bias Effect Chad Dube and Caren M. Rotello University of Massachusetts Evan Heit University of California, Merced A belief bias effect in syllogistic reasoning (Evans, Barston, & Pollard, 1983) is observed when subjects accept more valid than invalid arguments and more believable than unbelievable conclusions and show greater overall accuracy in judging arguments with unbelievable conclusions. The effect is measured with a contrast of contrasts, comparing the acceptance rates for valid and invalid arguments with believable and unbelievable conclusions. We show that use of this measure entails the assumption of a threshold model, which predicts linear receiver operating characteristics (ROCs). In 3 experiments, subjects made “valid”/”invalid” responses to syllogisms, followed by confidence ratings that allowed the construction of empirical ROCs; ROCs were also constructed from a base-rate manipulation in one experiment. In all cases, the form of the empirical ROCs was curved and therefore inconsistent with the assumptions of Klauer, Musch, and Naumer’s (2000) multinomial model of belief bias. We propose a more appropriate, signal detection– based model of belief bias. We then use that model to develop theoretically sound and empirically justified measures of decision accuracy and response bias; those measures demonstrate that the belief bias effect is simply a response bias effect. Thus, our data and analyses challenge existing theories of belief bias because those theories predict an accuracy effect that our data suggest is a Type I error. Our results also provide support for processing theories of deduction that assume responses are driven by a graded argument-strength variable, such as the probability heuristic model proposed by Chater and Oaksford (1999). Keywords: belief bias, signal detection, multinomial modeling, syllogistic reasoning, response bias In a recent ruling by the British Court of Appeals, judges concluded that, contrary to the ruling of a lower court, Pringles are in fact potato chips (Associated Press, 2009). One potential result of this decision is the application of the British value-added tax to Pringles. Much of the argument concerning the chip status of the Pringle hinged on the opinions of high authorities as to whether the percentage of potato content in a Pringles chip (actually less than 50%) was large enough to classify it as a potato chip. The logical structure of the argument can be analyzed by arranging the information as follows. Some of a Pringle is sliced potatoes. Some sliced potatoes are potato chips. ______________________________ A Pringle is a potato chip. (A) This argument is not logically valid, but the conclusion may be compelling to Pringles fans. The tendency to accept or reject a conclusion on the basis of its consistency with everyday knowledge, regardless of its logical status, is known as belief bias (e.g., Cherubini, Garnham, Oakhill, & Morley, 1998; Evans, Handley, & Harper, 2001; Evans, Newstead, & Byrne, 1993; Markovits & Nantel, 1989; Roberts & Sykes, 2003; Shynkaruk & Thompson, 2006). Belief bias is typically studied using categorical syllogisms, which are similar in structure to the argument in Example A. Syllogisms contain two premises and a conclusion, constructed with three terms: the predicate (X), which is the nonre- peated term of the first premise; the middle term (Y); and the subject (Z), which is the nonrepeated term of the second premise. An allow- able conclusion links the subject and predicate terms via their rela- tionship to the middle term; it may be either valid (following neces- sarily from the premises) or invalid. An abstract example of a valid syllogism is given in Example B: All X are Y. No Y are Z. _________ No Z are X. (B) The arrangement of the premise terms is referred to as the syllogistic figure. Four arrangements are possible: Y-X, Z-Y (Syl- logistic Figure 1); X-Y, Z-Y (Syllogistic Figure 2); Y-X, Y-Z (Syllogistic Figure 3); X-Y, Y-Z (Syllogistic Figure 4). The argu- ment in Example B is an example of a syllogism in Syllogistic Figure 4. Traditionally, each premise of the syllogism can take one of four quantifiers: “all,” “no,” “some,” and “some . . . are not.” Chad Dube and Caren M. Rotello, Department of Psychology, Univer- sity of Massachusetts; Evan Heit, School of Social Sciences, Humanities and Arts, University of California, Merced. This research was supported by National Science Foundation Collabo- rative Research Grant BCS-0616979 to Evan Heit and Caren M. Rotello. Chad Dube was supported as a trainee on National Institutes of Health Grant T32 MH16745. We thank Lissette Alvarez, Brooklynn Edwards, Efferman Ezell, V. Karina Ibanez, Chanita Intawan, Markie Johnson, Nic Raboy, Haruka Swendsen, and Jonathan Vickrey for their assistance in this research. We also thank Arndt Bröder, Christoph Klauer, and Mike Oaks- ford for their comments. Correspondence concerning this article should be addressed to Chad Dube, Department of Psychology, Box 37710, University of Massachu- setts, Amherst, MA 01004-7710. E-mail: cdube@psych.umass.edu Psychological Review © 2010 American Psychological Association 2010, Vol. 117, No. 3, 831– 863 0033-295X/10/$12.00 DOI: 10.1037/a0019634 831