10 © 2006 The Authors. Journal compilation © 2006 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice 13 (2007) 10–15 Journal of Evaluation in Clinical Practice ISSN 1356-1294 Blackwell Publishing LtdOxford, UKJEPJournal of Evaluation in Clinical Practice1356 1294© 2006 The Authors Journal compilation © 2006 Society/Blackwell Publishing Ltd2006131015Original Article Evaluating psychodiagnostic deci sion makingC.L.M. Witteman et al . Evaluating psychodiagnostic decisions Cilia L. M. Witteman MPhil PhD, 1 Clare Harries PhD CPsychol, 2 Hilary L. Bekker MSc PhD CHealthPsychol 3 and Edward J. M. Van Aarle MSc PhD CPedagogue 4 1 Professor of Psychodiagnostic Decision Making, 4 Lecturer in Psychodiagnosis, Behavioural Science Institute, Radboud University, Nijmegen, the Netherlands 2 Lecturer in Psychology, Department of Psychology, University College London, London, UK 3 Senior Lecturer in Behavioural Sciences, Academic Unit of Psychiatry and Behavioural Sciences, University of Leeds, Leeds, UK Abstract Rationale Several frameworks can be used to evaluate decision making. These may relate to different aspects of the decision-making process, or concern the decision outcome. Evaluations of psychodiagnostic decisions have shown diagnosticians to be poor decision makers. In this essay we argue that this finding results from the evaluation of only one part of the diagnostic process. Method We put forward that evaluations are typically carried out by comparing clinicians’ behaviour to one of several normative models, for example hypothetico-deductive reason- ing. These models make strong assumptions about human reasoning capabilities, which make it almost impossible for people to measure up to them. The subsequent two parts of the psychodiagnostic process (causal explanation and treatment decisions), are typically not included in these evaluation studies. Treatment decisions are evaluated in effectiveness studies; that is, they are evaluated in terms of their outcomes, not in terms of the diagnosti- cians’ decision processes. Psychodiagnosticians’ causal explanation has hardly ever been the subject of evaluation. We argue that in order to achieve clinical excellence, this part of the psychodiagnostic process should also be well understood. Results and Conclusions In this essay we first describe evaluation of psychodiagnostic decision making. We then propose a framework to describe causal explanation, that is, a situation assessment in terms of a causal schema or a story or script. We identify and discuss the tools available for evaluating this part of the psychodiagnostic process. Introduction When Di Caccavo & Reid [1] asked 36 General Practitioners how they made patient management decisions, five of them indicated that they did not have access to their own decision processes, stating for example ‘. . . it all goes into a dark box and makes a decision – which sometimes mystifies me . . .’ Eleven indicated the same but more subtly: they explained how they negotiated management decisions to their patients, once they themselves had already decided on courses of action. Twelve gave general expla- nations, talking in terms of tailoring the decision to the patients’ needs and expectations or using their medical training and experi- ence. Eight of the doctors gave scientific theories about their decision making, including references to pattern recognition, algo- rithms, decision trees and personal protocols. Just as these General Practitioners did, researchers in judge- ment, decision making and reasoning, coming from different disciplines within and outside psychology have used several theo- retical frameworks to describe and evaluate clinicians’ cognitive behaviour [2]. For example, diagnostic decision making has been framed as using exemplars, prototypes or forceful-feature driven pattern recognition (e.g. Genero & Cantor [3]; Elstein [4]); as forward or backward logical reasoning (e.g. Patel & Groen [5]), as hypothesis testing (e.g. Elstein et al . [6,7]) and as statistical reasoning (e.g. Eva & Brooks [8]); as information integration and use (e.g. Harries et al . [9]; Dhami & Harries [10]); as reasons- based [11]; as story [12,13] or script based [14–16]; as based on attitudes and beliefs [17]; and as based on the use of heuristics [18]. From one perspective such theoretical frameworks coexist because they are simply different rhetorical tools: paramorphic descriptions of the same underlying process, as are all models [19]. The same clinician’s behaviour might be framed as pattern recognition, as story-telling, or as combining information, depending upon the particular interests of the researcher and the researcher’s audience. From another perspective, different descriptions may refer to the behaviours of clinicians of different levels of experience, who are experiencing different task condi- tions, or who perform different sub-tasks of the diagnostic process [20,21]. For example, Einhorn et al. suggest that a think aloud Keywords causal explanations, clinical decision making, hypothetico-deductive reasoning, psychodiagnosis, scripts and stories Correspondence Dr Cilia Witteman Diagnostic Decision Making Behavioural Science Institute Radboud University PO Box 9104 (6th) 6500 HE Nijmegen the Netherlands E-mail: c.witteman@socsci.ru.nl Accepted for publication: 9 November 2005 doi:10.1111/j.1365-2753.2006.00689.x