Psychometric Evaluation of the Internalized Stigma of Mental Illness Scale for Patients with Mental Illnesses: Measurement Invariance across Time Chih-Cheng Chang 1,2,3 , Tsung-Hsien Wu 4 , Chih-Yin Chen 3,5 , Jung-Der Wang 6,7 , Chung-Ying Lin 6 * 1 Department of Psychiatry, Chi Mei Medical Center, Tainan, Taiwan, 2 Health Service and Population Research Department, Institute of Psychiatry, King’s College London, London, United Kingdom, 3 Department of Senior Citizen Service Management, College of Recreation and Health Management, Chia Nan University of Pharmacy and Science, Tainan, Taiwan, 4 Department of Psychiatry, Chi Mei Medical Center, Liouying, Tainan, Taiwan, 5 Department of Nursing, College of Health Sciences, Chang Jung Christian University, Tainan, Taiwan, 6 Deparment of Public Health, College of Medicine, National Chen Kung University, Tainan, Taiwan, 7 Departments of Internal Medicine and Occupational and Environmental Medicine, National Cheng Kung University Hospital, Tainan, Taiwan Abstract Background: The current investigation examined the psychometric properties of the Internalized Stigma of Mental Illness (ISMI) scale in a sample of patients with mental illness. In addition to the internal consistency, test-retest reliability, and concurrent validity that previous studies have tested for the ISMI, we extended the evaluation to its construct validity and measurement invariance using confirmatory factor analysis (CFA). Methods: Three hundred forty-seven participants completed two questionnaires (i.e., the ISMI and the Depression and Somatic Symptoms Scale [DSSS]), and 162 filled out the ISMI again after 50.23631.18 days. Results: The results of this study confirmed the frame structure of the ISMI; however, the Stigma Resistance subscale in the ISMI seemed weak. In addition, internal consistency, test-retest reliability, and concurrent validity were all satisfactory for all subscales and the total score of the ISMI, except for Stigma Resistance (a = 0.66; ICC = 0.52, and r = 0.02 to 0.06 with DSSS). Therefore, we hypothesize that Stigma Resistance is a new concept rather than a concept in internalized stigma. The acceptable fit indices supported the measurement invariance of the ISMI across time, and suggested that people with mental illness interpret the ISMI items the same at different times. Conclusion: The clinical implication of our finding is that clinicians, when they design interventions, may want to use the valid and reliable ISMI without the Stigma Resistance subscale to evaluate the internalized stigma of people with mental illness. Citation: Chang C-C, Wu T-H, Chen C-Y, Wang J-D, Lin C-Y (2014) Psychometric Evaluation of the Internalized Stigma of Mental Illness Scale for Patients with Mental Illnesses: Measurement Invariance across Time. PLoS ONE 9(6): e98767. doi:10.1371/journal.pone.0098767 Editor: Nori Takei, United (Osaka U, Kanazawa U, Hamamatsu U Sch Med, Chiba U and Fukui U) Graduate School of Child Development, Japan Received January 6, 2014; Accepted May 7, 2014; Published June 2, 2014 Copyright: ß 2014 Chang et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: This study was supported by grant CLFHR10028 from the Chi Mei Medical Center, Liouying, Taiwan. This research was, in part, supported by the Ministry of Education, Taiwan, R.O.C.; and the Aim for the Top University Project to the National Cheng Kung University (NCKU). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. * E-mail: cylin36933@gmail.com Introduction The stigma of mental illness, unlike the stigma of other medical conditions (e.g., epilepsy, leprosy, and cancer), is still highly prevalent in most high-income countries with good healthcare [1,2]. People with a mental illness are easily discriminated against because of the negative stereotypes attached to mental illness [3]. Therefore, the stigma they face erodes their social status, interpersonal relationships, quality of life, and self-esteem [4–6]. As a result, people with mental illness are at a high risk of unemployment, isolation, and delayed treatment-seeking, which often causes a serious public health burden. One aspect of the stigma is internalized stigma (self-stigma), discussion of which has increased over the past few decades [2]. Because people with mental illness encounter external and objective discrimination, such as reduced access to employment and housing, they may translate this discrimination into self- devaluation. Thus, people with mental illness are left feeling that they are not members of the society in which they live [7]. In addition, even those who have not experienced discrimination (the behavioral manifestation of public stigma) may also feel alienated because of prejudice (the attitudinal manifestation of public stigma) [8]. Ritsher (Boyd) et al. [7] call this kind of stigma ‘‘internalized stigma’’, and say that it is ‘‘one of the especially painful and destructive effects of stigma’’. Based on their definition, Ritsher (Boyd) et al. [7] developed a reliable and valid instrument, the Internalized Stigma of Mental Illness (ISMI) scale for people with mental illness, and the ISMI has been broadly used in different cultures [9]. The ISMI is useful for clinicians, and this validated measure encourages researchers and clinicians to use, in addition to symptom reduction, self-stigma reduction as a concurrent treat- PLOS ONE | www.plosone.org 1 June 2014 | Volume 9 | Issue 6 | e98767