Journal of Statistical Planning and Inference 111 (2003) 209 – 221 www.elsevier.com/locate/jspi Posterior predictive model checking in hierarchical models Sandip Sinharay a;1 , Hal S. Stern b; ∗ a Educational Testing Service, Princeton, NJ, USA b Department of Statistics, 4900H Berkeley Place, University of California, Irvine, CA 92697, USA Abstract Model checking is a crucial part of any statistical analysis. Hierarchical models present special problems because assumptions made about the distribution of unobservable parameters are di- cult to check. In this article, we review some approaches to model checking and apply posterior predictive model checking to a hierarchical normal–normal model analysis of data from educa- tional testing experiments in eight schools. Then we carry out a simulation study to investigate the diculties in model checking for hierarchical models. It turns out that it is very dicult to detect violations of the assumptions made about the population distribution of the parameters unless the extent of violation is huge or the observed data have small standard errors. c  2002 Elsevier Science B.V. All rights reserved. MSC: primary 62F15 Keywords: Discrepancy; Marginal model; Posterior predictive p-value; Random eects model 1. Introduction Assessing the validity of model assumptions is a crucial part of any parametric statistical analysis. Hierarchical models, in which data y are modeled conditional on a collection of parameters  and these parameters are in turn described by a probability distribution with underlying parameters , present special problems because assumptions made about the distribution of the unobservable parameters are dicult to check. As hierarchical models are increasingly popular in applications in the health sciences, animal breeding and many other elds (see, e.g., Gelman et. al., 1995; Carlin and Louis, 1996; Gilks et. al., 1996), the development of tools and techniques for model * Corresponding author. Tel.: +949-824-1568. E-mail address: sternh@uci.edu (H.S. Stern). 1 Associate Research Scientist. 0378-3758/02/$ - see front matter c  2002 Elsevier Science B.V. All rights reserved. PII:S0378-3758(02)00303-8