Experiences of Collaborative Research Daniel Kahneman Princeton University The author’s personal history of the research that led to his recognition in economics is described, focusing on the process of collaboration and on the experience of contro- versy. The author’s collaboration with Amos Tversky dealt with 3 major topics: judgment under uncertainty, decision making, and framing effects. A subsequent collaboration, with the economist Richard Thaler, played a role in the development of behavioral economics. Procedures to make controversies more productive and constructive are suggested. The Collaboration With Amos Tversky I t was the spring of 1969, and I was teaching a graduate seminar at the Hebrew University of Jerusalem on the applications of psychology to real-world problems. In what turned out to be a life-changing event, I asked my younger colleague Amos Tversky to tell the class about what was going on in his field of judgment and decision making. Amos told us about the work of his former mentor, Ward Edwards, whose lab was using a research paradigm in which participants were shown two book bags filled with poker chips. The bags were said to differ in their composition (e.g., 70 red and 30 white chips or 30 red and 70 white chips). One bag was randomly chosen. The participant was given an opportunity to sample successively from it and was required to indicate after each trial the probability that it came from the predominantly red bag. Edwards had concluded from the results that people are conservative Bayesians: They almost always adjust their confidence interval in the proper direction, but rarely far enough. A lively discussion developed around Amos’s talk. The idea that people were conservative Bayesians did not seem to fit with the everyday observation of people com- monly jumping to conclusions. It also appeared unlikely that the results obtained in the sequential sampling para- digm would extend to the situation, arguably more typical, in which evidence is delivered all at once. Finally, the label conservative Bayesian suggested a process that first gets the correct answer, then adulterates it with a bias—not a plausible psychological mechanism. I learned recently that one of Amos’s friends met him that day and heard about our conversation, which Amos described as having se- verely shaken his faith in the neo-Bayesian approach. Amos and I decided to meet for lunch to discuss our hunches about the manner in which probabilities are “re- ally” judged. There we exchanged personal accounts of our own recurrent errors of judgment in this domain and de- cided to study the statistical intuitions of experts. I spent the summer of 1969 doing research at the Applied Psychological Research Unit in Cambridge, En- gland. Amos stopped there for a few days on his way to the United States. I had drafted a questionnaire on intuitions about sampling variability and statistical power, which was based largely on my personal experiences of incorrect research planning and unsuccessful replications. The ques- tionnaire consisted of a set of problems, each of which could stand on its own, making its own point—I had long had the ambition of doing “psychology with single ques- tions,” and this was an opportunity to try it. Amos went off and administered the questionnaire to participants at a meeting of the Mathematical Psychology Association, and a few weeks later, we met in Jerusalem to look at the results and write a paper. The experience was magical. I had enjoyed collabo- rative work before, but this was different. Amos was often described by people who knew him as the smartest person they knew. He was also very funny, with an endless supply of jokes appropriate to every nuance of a situation. In his presence, I became funny as well, and the result was that we could spend hours of solid work in continuous mirth. The paper we wrote was deliberately humorous—we de- scribed a prevalent belief in the “law of small numbers,” according to which the law of large numbers extends to small numbers as well. Although we never wrote another humorous paper, we continued to find amusement in our work—I have probably shared more than half of the laughs of my life with Amos. And we were not just having fun. I quickly discovered that Amos had a remedy for everything I found difficult about writing. With him, movement was always forward. Progress might be slow, but each successive draft we produced was an improvement—this was not something I could take for granted when working on my own. Amos’s work was always characterized by confidence and by a crisp elegance, and it was a joy to find those characteristics now attached to my ideas as well. As we were writing our first paper, I was conscious of how much better it was than the more hesitant piece I would have written by myself. I Editor’s note. This excerpt was adapted with permission from a longer autobiographical statement to be published in Les Prix Nobel 2002 (Fra ¨ngsmyr, in press). Copyright by the Nobel Foundation. Adapted by permission. Author’s note. Correspondence concerning this article should be ad- dressed to Daniel Kahneman, Department of Psychology, Princeton Uni- versity, Princeton, NJ 08544-1010. E-mail: kahneman@princeton.edu 723 September 2003 ● American Psychologist Copyright 2003 by the American Psychological Association, Inc. 0003-066X/03/$12.00 Vol. 58, No. 9, 723–730 DOI: 10.1037/0003-066X.58.9.723