Handbook of Statistics, Vol. 25 ISSN: 0169-7161 © 2005 Elsevier B.V. All rights reserved. DOI 10.1016/S0169-7161(05)25005-8 5 Role of P-values and other Measures of Evidence in Bayesian Analysis Jayanta Ghosh a , Sumitra Purkayastha and Tapas Samanta Abstract We review the relation between P-values and posterior probabilities, and also the Bayesian P-values for composite null hypothesis when there is no alternative. Keywords: Bayes factors; Bayesian P-values; Pitman alternative; posterior proba- bility 1. Introduction A P-value is “the smallest level at which the observations are significant in a partic- ular direction” (Gibbons and Pratt, 1975). Equivalently, it is the probability, under a simple null or the supremum of probabilities under a composite null, of deviations of the test statistic being greater than that currently observed. As a data based measure of evidence against the null hypothesis H 0 , it contains more information than just a statement of significance or lack of it at a predetermined level of significance, or the Neyman–Pearsonian probability of Type I error. Therein lies its attraction as a measure of evidence in classical statistical methodology for testing. Gibbons and Pratt (1975) argue that in “any single experiment the P-value measures the agreement or disagreement in a specific direction” and that, therefore, it serves a useful purpose. On the other hand they point out that one cannot compare P-values across sample sizes or across experiments – a point also made very effectively in Lindley (1957). It is also clear and has been noted many times that the frequentist interpretation of a P-value under a simple null is somewhat odd in that it depends on hypothetical repetition leading to new samples which have more deviant values of a test statistic than the value actually observed. Among other things it implies an apparent use of unobserved more deviant data as evidence. A fact which is both an advantage and a disadvantage is that a P-value only depends on the distribution under the null hypothesis, it ignores the alternative. This is an advantage because the null distribution is usually a Corresponding author. 151