Commenting on Comments Moderator: Nancy Dudek, MD Discussant: Joan Sargeant, PhD Which Factors, Personal or External, Most Influence Students’ Generation of Learning Goals? Kevin W. Eva, Juan Munoz, Mark D. Hanson, Allyn Walsh, and Jacqueline Wakefield Abstract Background While concern has been expressed about the validity of self-assessments, external feedback is likely filtered through self- assessment. This paper explores the relationship between self-assessments and feedback uptake. Method During an objective structured clinical examination, students were asked to evaluate their performance and rate the quality of feedback provided by observers. Afterward, they were asked to list learning goals they generated, to indicate what activities they would undertake to fulfill those goals, and to identify which station(s) led them to generate each response. Regression analyses were used to determine which variables predicted the generation of goals/activities. Results Students’ perceptions of their own performance were more likely to result in the generation of goals/strategies than was observer feedback or student perceptions of observer feedback quality. Later stations were more likely to result in goal/strategy generation than earlier stations. Conclusions While self-assessments may not validly indicate ability, it is still critical to determine how students perceive their ability because their opinions drive their learning goals. Not a day passes without each of us making decisions that are influenced by our impressions of our own skill, knowledge, or potential. By choosing to ride a bike to work, one expresses confidence that he is able to do so without experiencing paralysis (by exhaustion or through a worse fate). By applying to a health professional training program, one expresses the expectation that she has the mental and interpersonal capacity to learn to perform well in her desired career. By prioritizing one’s professional development reading, one expresses the belief that there is more to be gained by studying one topic relative to another. On each occasion, countless sources of information contribute to the judgments made: Examples include past success (e.g., having remained upright on one’s bike previously), one’s ego, and the feedback received from other individuals regarding what aspects of performance could or should be improved. It is easy to make an argument, as a result, that the individual is better positioned than anyone else to make judgments about his or her personal strengths, weaknesses, and learning needs. Indeed, this is the very foundation on which many models of professional regulation have been built; individual practitioners are routinely charged with determining how to best continue their professional development within the context of their specific and idiosyncratic personal practice profile. 1 Unfortunately, however, it has become equally easy to argue that the various sources of information on which one has to draw are often not mentally aggregated in a manner that engenders an accurate impression of one’s abilities. 2 Self- assessment, defined as an individually generated summary judgment of one’s skill level, is clearly insufficient for judging ability, despite findings that suggest we can judge our likelihood of success with a specific problem at a specific moment in time. 3,4 This literature has prompted many to argue that external feedback is essential to help guide performance improvements, be that feedback delivered in the form of testing, data pertaining to one’s practice, or guidance provided by a colleague/supervisor. 5 Herein, however, an interesting paradox is beginning to emerge. Personal experiences do not appear sufficient for the creation of accurate self-assessments; this leads to calls for better feedback— yet, such feedback must be delivered and received within the context of the flawed self-assessments. That is, feedback never occurs in a vacuum and, as such, will always be interpreted in the context of one’s impressions of one’s own ability, regardless of the trustworthiness of those impressions. Indeed, research performed on the impact of multisource feedback has suggested that the perceived value of feedback depends on the ease with which the feedback can be reconciled with one’s self-assessment. 6 In an effort to move toward a better understanding of the relationship between self-assessments and the uptake of external feedback, this paper presents a study that was aimed at addressing the question, Which factors, personal or external, most influence students’ generation of learning goals? It goes without saying that goals are not always achieved, but there is a clear relationship between the establishment of goals and pedagogical benefit. 7 We chose to study this issue in the context of an objective structured clinical examination (OSCE) because (1) our students have anecdotally reported that they perceive the OSCE to be a particularly beneficial educational Correspondence: Kevin W. Eva, PhD, JPPN 3324, 910 West Tenth Avenue, University of British Columbia, Vancouver, BC, V5Z 4E3, Canada; e-mail: kevin.eva@ubc.ca. Acad Med. 2010;85:S102–S105. doi: 10.1097/ACM.0b013e3181ed42f2 Academic Medicine, Vol. 85, No. 10 / October 2010 Supplement S102