Effect Size Indices for Artificially Dichotomized Variables Measured with Error: An Empirical Investigation of Accuracy and Precision Isaac Li 1 , Patricia Rodríguez de Gil 1 , Jeanine Romano 1 , Aarti P. Bellara 1 , George MacDonald 1 , Harold Holmes 1 , Patrice Rasmussen 1 , Yi-Hsin Chen 1 , Jeffrey D. Kromrey 1 1 Educational Measurement and Research, University of South Florida, College of Education, 4202 East Fowler Avenue, EDU 105, Tampa, FL 33620 Abstract Monte Carlo methods were used to investigate the accuracy and precision of effect size indices in estimating what the standardized mean difference from a 2 X 2 sample table of dichotomized variables would have been had the data not been dichotomized. Normally distributed, continuous data were generated for two groups and the continuous variable was dichotomized at specified cut points. The factors manipulated in the simulation study included overall sample size (n 1 + n 2 = 30, 60, 120, 240), reliability levels (.5, .7, .8, .9, 1), population effect size (0, .2, .5, .8), continuous score cut point for dichotomization (.10, .25, .40, .50, .70), and population variance ratio (1:1, 1:2, 1:4). For each sample generated (100,000 replications), each of seven proposed effect size indices was calculated. Both the statistical bias and the RMSE were computed across the set of replications. Although the sample standardized mean difference became substantially biased in the presence of measurement error, the performance of the seven indices was not notably affected. Results were interpreted in terms of recommendations for estimating effect sizes with dichotomized variables. Key Words: effect sizes, simulation, reliability, statistical bias, dichotomy 1. Background The topic of effect sizes can be a controversial issue for many journal editors, as well as for researchers. For instance, Pedhazur and Pedhazur-Schmelking (1991) argued that Cohen’s (1988) convention of small, medium, and large effect sizes distorted the distinction between the magnitude of an effect and its substantive importance, i.e., researchers relegating “small” effects as less important or considering “large” effects as important. Also, the use and interpretation of a specific effect size across studies can be problematic due to the variability of research design factors, the last edition of the Publication Manual of the American Psychological Association (6th edition; 2010) as well as the 1999 report by Wilkinson and the APA Task Force on Statistical Inference have made clear the imperative for reporting effect sizes to supplement statistical hypothesis testing and ensure “accuracy of scientific knowledge” (p. 11). Effect sizes provide useful indices of the magnitudes of treatment effects in individual studies as well as representing the primary statistics that are used in synthesizing research or meta- analysis. Cohen (1977) defined effect size (ES) as “the degree to which the phenomenon is present in the population” (pp. 9-10), or “the degree to which the H 0 is believed to be false” JSM 2013 - Social Statistics Section 2403