l~ychologicalReview Copyright 1987 by the American Psychological Association, Inc. 1987, Vol. 94, No. 2, 211-228 0033-295X/87/$00.75 Confirmation, Disconfirmation, and Information in Hypothesis Testing Joshua Klayman and Young-Won Ha Center for Decision Research, Graduate School of Business, University of Chicago Strategies for hypothesis testing in scientific investigation and everyday reasoning have interested both psychologists and philosophers. A number of these scholars stress the importance of disconfir. marion in reasoning and suggestthat people are instead prone to a general deleterious "confirmation bias" In particula~ it is suggested that people tend to test those cases that have the best chance of verifying current beliefs rather than those that have the best chance of falsifying them. We show, howeve~ that many phenomena labeled "confirmation bias" are better understood in terms of a general positive test strate~. With this strategy, there is a tendency to test cases that are expected (or known) to have the property of interest rather than those expected (or known) to lack that property. This strategy is not equivalent to confirmation bias in the first sense; we show that the positive test strategy can be a very good heuristic for determining the truth or falsity of a hypothesis under realistic conditions~ It can, howeve~ lead to systematic errors or inefficiencies. The appropriateness of human hypotheses-testingstrategies and prescriptions about optimal strategies must he under- stood in terms of the interaction between the strategy and the task at hand. A substantial proportion of the psychological literature on hypothesis testing has dealt with issues of confirmation and dis- confirmation. Interest in this topic was spurred by the research findings of Wason (e.g., 1960, 1968) and by writings in the phi- losophy of science (e.g., Lakatos, 1970; Platt, 1964; Popper, 1959, 1972), which related hypothesis testing to the pursuit of scientific inquiry. Much of the work in this area, both empirical and theoretical, stresses the importance of disconfirmation in learning and reasoning. In contrast, human reasoning is often said to be prone to a "confirmation bias" that hinders effective learning. Howeve~ confirmation bias has meant different things to different investigators, as Fischboff and Beyth-Marom point out in a recent review (1983). For example, researchers studying the perception of correlations have proposed that people are overly influenced by the co-occurrence of two events and in- suiticiently influenced by instances in which one event occurs without the other (e.g., Arkes & Harkness, 1983; Crocker; 1981; Jenkins & Ward, 1965; Nisbett & Ross, 1980; Schustack & Sternberg, 1981; Shaldee & Mires, 1982; Smedslund, 1963; Ward & Jenkins, 1965). Other researchers have suggested that people tend to discredit or reinterpret information counter to a hypothesis they hold (e.g., Lord, Ross, & Leppe~ 1979; Nisbett & Ross, 1980; Ross & Leppe~ 1980) or they may conduct biased tests that pose little risk of producing disconfirming results This work was supported by Grant SES-8309586 from the Decision and Management Sciences program of the National Science Founda- tion. We thank Hillel Einhom, Ward Edwards, Jackie Gnepp, William Goldstein, Steven Hoch, Robin Hogarth, George Loewenstein, Nancy Pennington, Jay Russo, Paul Schoemaker, William Swann, Tom Tra- basso, Ryan Tweney, and three anonymous reviewers for invaluable comments on earlier drafts. Correspondence concerning *~his article should be addressed to Joshua Klayman, Graduate School of Business, University of Chicago, 1101 East 58th Street, Chicago, Illinois 60637. 211 (e.g., Snyder, 1981; Snyder & Campbell, 1980; Snyder & Swarm, 1978). The investigation of hypothesis testing has been concerned with both descriptive and prescriptive issues. On the one hand, researchers have been interested in understanding the processes by which people form, test, and revise hypotheses in social judg- ment, logical reasoning, scientific investigation, and other do- mains. On the other hand, there has also been a strong implica- tion that people are doing things the wrong way and that efforts should be made to correct or compensate for the failings of hu- man hyix~thesis testing. This concern has been expressed with regard to everyday reasoning (e.g., see Bruner, 1951; Nisbett & Ross, 1980) as well as professional scientific endeavor (e.g., Mahoney, 1979; Plan, 1964). In this article, we focus on hypotheses about the factors that predict, explain, or describe the occurrence of some event or property of interest. We mean this broadly, to include hypothe- ses about causation ("Cloud seeding increases rainfall"), cate- gorization ("John is an extrovert"), prediction ("The major risk factors for schizophrenia are..."), and diagnosis ("The most diagnostic signs of malignancy are.. :'). We consider both de- scriptive and prescriptive issues concerning information gather- ing in hypothesis-testing tasks. We include under this rubric tasks that require the acquisition of evidence to determine whether or not a hypothesis is correct. The task may require the subject to determine the truth value of a given hypothesis (e.g., Jenkins & Ward, 1965; Snyder & Campbell, 1980; Wason, 1966), orto find the one true hypothesis among a set or universe ofpossibilities (e.g., Bruner, Goodnow, & Austin, 1956; Mynatt, Doherty, & Tweney, 1977, 1978; Wason, 1960, 1968). The task known as rule discovery (Wason, 1960) serves as the basis for the development of our analyses, which we later extend to other kinds of hypothesis testing. We first examine what "confirmation" means in hypothesis testing. Different senses of confirmation have been poorly distinguished in the literature, contributing to misinterpretations of both empirical findings