104 COMMENTARY Potential for Bias of Data From Functional Status Measures Mary Ann Plant, PhD, J. Scott Richards, PhD, Nancy K. A FUNNY THING is happening on the way to a prospective payment system based on functionally related groups (FRGs): those knowledgeable in the field of functional status measures (FSMs) have acknowledged that maintaining the in- tegrity of the data that form the foundation may be problem- atic. TM Nevertheless, since functional status measures often re- flect the primary endpoint of rehabilitation charges and/or length of stay, they have been recommended by many authors as com- ponents of a potential payment model or recommended with some reservations and precautions. 4-7 Several FSMs, both long- established and newly developed tools, are being investigated for their reliability and validity. TM There are also some initial efforts to establish normative data on functional status and change in functional status within impairment categories. 9'16 However, with but a few exceptions, 17 there has been little research devoted to identifying and empirically describing the potential threats to the FSM data-gathering process itself. The Functional Independence Measure (FIM) is currently the most widely used of the FSMs; its developers have produced proto- cols for training raters to criterion initially and recertifying raters periodically to maintain high interrater reliabilityJ °'~8 Despite the demonstrated capacity for high interrater agreement on the FIM when rating a patient at one point in time, some critics have identified a vulnerability of the FIM and similar FSMs to "gaming," or intentional manipulation of ratings to market patient status or progressJ 9 Payment models based on progress measured by FSMs increase the probability that providers will game the system and even create such a temptation because providers have professional economic self-interests in recording functional gainsJ Thus, staff may be faced with conflicting goals when rating patient status: accuracy versus the pressure to show optimal outcomes. In addition to deliberate manipulation of data, the possibility exists for unintentional bias in judgments regarding functional status. Although people think of their own logic and decision- making as objective and free from bias, a considerable body of research in cognitive and social psychology indicates that perceptions and judgments are often biased according to com- mon but mistaken "rules" of thinking, called heuristics. 2°'2] Researchers in the rehabilitation field have begun to recog- nize the possibility that such biases and heuristics might influ- ence professional judgments in rehabilitation. For example, in From the Adult and Child DevelopmentProfessionals (Dr. Plant), Birmingham, AL; the Departmentof PhysicalMedicineand Rehabilitation, University of Alabama at Birmingham(Dr. Richards); and the Department of Psychology, Saginaw Valley State University(Dr. Hansen), Flint, MI. Dr. Hansen is now associated with the Institute of Psychology,IllinoisInstitute of Technology,Chicago. Submitted for publication May 12, 1997. AcceptedJune 26, 1997. The views expressed in this article are those of the authors and do not necessar- ily reflectthe views of the University of Alabama at Birmingham,Saginaw Valley State University, or any other organization with which the authors are affiliated. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are affiliated. Reprint requests to J. Scott Richards, PhD, Spain Rehabilitation Center, Room 530, 1717 6th Avenue South, Birmingham, AL 35233. © 1998 by the AmericanCongressof RehabilitationMedicineand the American Academy of Physical Medicine and Rehabilitation 0003-9993/98/7901-451653.00/0 Hansen, PhD a meta-analysis of rehabilitation outcomes research, results re- corded by observers who knew patients' treatment status tended to be higher than outcomes recorded by blind observers. 22 Blind- ing of raters to treatment group is taken for granted in research studies as a guarantee of the integrity of data; grant proposals to evaluate treatment outcomes in rehabilitation are expected to include such safeguards. The precaution of blinding FSM raters has been recommended as a check on potential gaming, l but has rarely been implemented in clinical settings. Self-serving bias, the tendency to attribute one's own suc- cesses to personal characteristics, but one's failures to external circumstances, has been noted in research by Macchiochi and EatonJ 7 In their study, therapists who were surveyed regarding their explanations for neurorehabilitation outcomes did not spontaneously mention the single most predictive variable (in- jury severity), and tended to provide therapist-related explana- tions for good outcomes while providing non-therapist-related explanations (eg, patient motivation) for poorer outcomes. Such attribution biases could be reflected in FSM scoring patterns in real-world decision making. For example, an "up-coding" at discharge might result from a self-serving bias regarding the efficacy of therapy provided when the raters are therapists who had participated in treatment. Similarly, a "down-coding" at admission would make any improvement by discharge seem even greater, as well as providing a "self-handicap" 23 to explain any failure of the patient to show gains after treatment. Other biases have been described that can lead to distortion and inaccuracy (eg, confirmation bias where initial opinion is subsequently always confirmed and contrary data are ig- nored). 2Z'24 Staff group discussion of patient data may increase bias, causing opinions to become more extreme either toward improvement or worsening, z5 In treatment or teaching situations a staff member may act according to expectations of the patient/ student, thereby actively contributing to the anticipated result-- whether positive or negative (self-fulfilling prophecy). 26'27 Decisions may also be influenced by the information avail- able in memory to the decision-maker or rater, creating an avail- ability bias) 1 Information is more likely to be remembered and available when it is vivid (because of recency, interest, or emo- tional charge) than when it is less vivid but perhaps more char- acteristic of the patient's functioning. Macchiochi and Eaton I7 demonstrated the importance of availability in clinical judgment by cuing raters to potentially important data or omitting cues. When raters were so cued, they did notice and utilize key infor- mation otherwise neglected. Information stored through profes- sional training appears to produce some bias; research indicates that disciplinary differences emerge in ratings on FSMs. 28'29 Another potential source of bias is missing data, ie, ratings not marked on FSM items. Study results from a variety of disci- plines indicate that severe bias may appear when such gaps in data sets are random; if item ratings are missing in some system- atic way, the distortion may be even more serious. 3° These briefly mentioned biases, while worth noting, are bidi- rectional, ie, thinking may be biased either for improvement or against it. The situation in rehabilitation outcomes seems more likely to stimulate directional bias in favor of improvement. Some of the biases and heuristics mentioned above have been shown to occur undonsciously and in the absence of external Arch Phys Med Rehabil Vol 79, January 1998