Journal of Administrative Sciences and Policy Studies December 2015, Vol. 3, No. 2, pp. 75-90 ISSN: 2372-5109 (Print), 2372-5117 (Online) Copyright © The Author(s). 2015. All Rights Reserved. Published by American Research Institute for Policy Development DOI: 10.15640/jasps.v3n2a5 URL: http://dx.doi.org/10.15640/jasps.v3n2a5 On Some Relative E ntropy Statistics E rhan Ustaoglu 1 & Atif E vren 2 Abstract Statistical entropy is a measure of dispersion or spread of a random variable. E specially when the random variable is nominal, classical measures of dispersion like standard deviation can not be computed. In such cases, measures of variation, including entropy-based statistics;computed by using cell frequencies of a distributionmust be used. The asymptotic properties of entropy statistics have long been studied in literature. Relative entropy plays an important role in evaluating the degree of fit. In other words, relative entropy is a measure of goodness fit of an empirical distribution to a theoretical or hypothesized distribution. In this study for some frequently-used probability distributions,some relative entropy measures are derived by exploiting additivity property of Kullback-Leibler divergence and Jeffreys divergence. Their asymptotic properties under certain assumptions have been discussed. In the end, by some applications, the close relation between relative entropy statistics and other classical test statistics have been emphasized. Keywords: Relative entropy, K ullback-Leibler divergence, Jeffreys divergence, mutual information, asymptotic properties of relative entropy Introduction Statistical entropy can be evaluated as ameasure of unpredictability of the outcome of a statistical experiment. The more predictable the outcome of an experiment, the less will be the uncertainty and soforth the entropy. After the experiment (or observation ) is carried out, the uncertainty is not present. So in some sense, entropy is a measure of information that one can get through statistical experimentation (Renyi, p23). 1 Marmara University, Faculty of Administrative Sciences, Department of Management, Information Sciences, Bahçelievler, 34180 Istanbul , Turkey. E -mail: erhan.ustaoglu@ gmail.com, tel: + 90 530 343 82 99 2 Yildiz Technical University, Faculty of Sciences and Literature, Department of Statistics Davutpasa, E senler, 34210, Istanbul, Turkey. E -mail: aevren@ yildiz.edu.tr, tel: + 90 533 3454973