Commentary on: Authors’ Response [J Forensic Sci 2009; 54(2):501] to Wells’ comments [J Forensic Sci 2009;54(2):500] regarding Krane DE, Ford S, Gilder JR, Inman K, Jamieson A, Koppl R, Kornfield IL, Risinger DM, Rudin N, Taylor MS, Thompson WC. Sequential unmasking: a means of minimizing observer effects in forensic DNA interpretation. J Forensic Sci 2008;53(4):1006–7. Sir, Let me say at the outset that I agree with the call by both Wells and Krane et al. for additional research into potential observer effects and ⁄ or bias in decision-making by forensic examiners. How- ever, in my opinion the situation is not nearly as clear as Krane et al. suggest in their response to Dr. Wells. In that response, eight studies are cited in the statement ‘‘Empirical studies have con- firmed that observer effects can influence [the results in various forensic disciplines].’’ My first impression upon reading this was that these studies showed that observer effects and ⁄ or bias were ‘‘proven’’ to be concerns in the listed disciplines. But one reference, in particular, caught my eye. The 1984 Miller (1) study may be familiar to some as it has been mentioned at least twice before; once at the 2008 AAFS meeting in Washington, DC (2) and again at a recent ‘‘Expert Forensic Evidence’’ conference in Toronto, ON (3). Despite its provocative title, this study provides absolutely no data that could be construed even remotely as per- taining to qualified forensic document examiners. Rather, ‘‘Twelve college students, trained in the forensic examination of questioned documents, were utilized in the experiment’’ (p. 409). No details were provided by the author about the nature of the training given to the students but I find it difficult to understand how the results of a study based entirely on college students can be extended to professional, qualified examiners in any meaningful manner. Upon seeing this reference in the list of citations I decided to review all of the studies. My review showed that these studies pro- vide limited, even ambiguous, data with respect to whether or not observer effects or bias are issues with qualified forensic examiners. Indeed, like the Miller study described above, three of the cited studies did not involve professional forensic examiners at all and instead used only students as test subjects. One study was a meta- analysis based upon two earlier studies. The remaining three studies provide rather conflicting data about the potential for bias ⁄ observer effects. I would encourage everyone to review these articles for themselves but a short discussion of each is provided here to clarify my position on this matter. Both of Miller’s studies from 1984 and 1987 (4) used students and included no trained examiners. In the 1987 study the author stated ‘‘Fourteen students enrolled in advanced crime laboratory college courses were selectively trained in human hair identification techniques. The training consisted of 60 academic hours of lecture and 60 academic hours of laboratory experience under the instruc- tion of court-qualified human hair experts. The 14 students met the basic requirements for expert testimony on human hair identifica- tion in courts of law. Each of the 14 examiners was independently advised to examine and compare human hair evidence in four crim- inal investigations’’ (p. 160). The author did not say if the students had successfully completed their courses; only that they were enrolled in such courses. The author’s assertion that the students ‘‘met the basic requirements for expert testimony’’ is open to inter- pretation. As most readers know, meeting the basic requirements to be qualified as a court expert may not, in fact, make someone qualified to do the work. Beyond this, even if the college students were considered to be advanced trainees (e.g., novice examiners) rather than naïve subjects, the extension of these results to fully qualified forensic examiners is dubious. The Beckham et al. study (5) in 1989 was quite extensive in that it involved 180 mental health experts who were asked to assess NGRI (Not Guilty by Reason of Insanity) submissions. Beckham et al. commented in part that ‘‘in the current study, no statistically significant bias was detected between groups…’’ (p. 86). In their discussion of this finding the authors further commented ‘‘Even though such bias has been demonstrated in clinical psychology graduate students, practicing forensic evaluators may be more attuned to such detrimental possibilities and therefore actively strive to be as objective as they can’’ (p. 86). If anything, this particular study suggests that bias was less of a problem than the researchers had anticipated though, of course, there is the issue of extension of the results to other types of forensic work. At any rate, it certainly does not support the belief that observer ⁄ bias effects are present in all types of forensic work. The Dror et al. study (6) in 2005 involved ‘‘…27 university stu- dent volunteers, with a mean age of 23 (9 were males and 18 were females). ’’ In their discussion of the results, the authors noted this limitation and commented ‘‘Second, our findings need to be exam- ined within the context of routine everyday work of fingerprint experts. The training, experience, and work procedures of finger- print experts may play an interesting and crucial role in if and how top-down components play a role in fingerprint identification. On the one hand, fingerprint experts may be less susceptible to top- down interference, perhaps even immune, to such effects. Given their highly specialized skills, they may be able to focus solely on the bottom-up component and be data driven without the external influences that we have observed in the research reported here. On the other hand, and in contrast, fingerprint experts may be even more susceptible to such top-down components’’ (pp. 807–808). Overall, these comments suggest to me a rather inconclusive posi- tion; a very reasonable position since the data from the study had nothing to do with qualified examiners. Following the 2005 study, two studies (7,8) from 2006 by Dror et al. used qualified fingerprint experts (five and six, respectively). These studies are arguably the most intriguing to date insofar as they provide some support for the belief that observer effects may influence fingerprint examiners in at least some situations. At the same time, the nature of the influence ⁄ bias effect is not entirely clear. Aside from the issue of generalization of results from rela- tively small sample sizes (a point discussed by the original authors), the bias effect seems to be mostly unidirectional. That is, bias attempts may shift some conclusions toward exclusion but they were not very successful in moving conclusions toward individuali- zation. In their Journal of Forensic Identification article, the authors speculated about this result saying ‘‘It seems that the threshold to make a decision of exclusion is lower than that to make a decision of individualization. Indeed our data support this claim, as reflected by the fact that most of the conflicting decisions were past individualizations. We did, however, observe a case in which an exclusion decision was now judged to be an individuali- zation. This relates to the decision-making model used by experts in the fingerprint domain’’ (p. 613). In the end, I would agree there is evidence to support the belief that observer effects can be a fac- tor in these types of comparisons but the precise nature, and the limits, of the influence is not clear from these studies. J Forensic Sci, November 2009, Vol. 54, No. 6 doi: 10.1111/j.1556-4029.2009.01191.x Available online at: interscience.wiley.com 1498 Ó 2009 American Academy of Forensic Sciences