Journal of Data Science 4(2006), 387-398 Data Information in Contingency Tables: A Fallacy of Hierarchical Loglinear Models Philip E. Cheng, Jiun W. Liou, Michelle Liou and John A. D. Aston Academia Sinica Abstract: Information identities derived from entropy and relative entropy can be useful in statistical inference. For discrete data analyses, a recent study by the authors showed that the fundamental likelihood structure with categorical variables can be expressed in different yet equivalent information decompositions in terms of relative entropy. This clarifies an essential differ- ence between the classical analysis of variance and the analysis of discrete data, revealing a fallacy in the analysis of hierarchical loglinear models. The discussion here is focused on the likelihood information of a three-way con- tingency table, without loss of generality. A classical three-way categorical data example is examined to illustrate the findings. Key words: Entropy, likelihood ratio test, loglinear models, mutual infor- mation. 1. Introduction The analysis of contingency tables with multi-way classifications originates from the historical development of statistical inference with 2 × 2 tables. In the initial extension to the case of 2 × 2 × K tables, Bartlett (1935) discussed test- ing for three-way interaction and derived an estimate of the common odds ratio suggested by R. A. Fisher. Norton (1945) and Simpson (1951) supplied inter- pretations of varied interactions which led to the well-known Simpson’s paradox (Blyth, 1972). Roy and Kastenbaum (1956) showed that Bartlett’s procedure is an implicit maximum likelihood estimation (MLE), conditioned upon the fixed margins of each 2 × 2 table. The celebrated analysis of variance (ANOVA, Fisher, 1925) inspired discussions of partitioning chi-squares within the contingency ta- bles, notably by Lancaster (1951), Mood (1950), and Claringbold (1961), among others. In related research in biostatistics, Cochran (1954), Woolf (1955) and Mantel and Haenszel (1959) developed chi-square tests for no association be- tween two variables across levels of the third variable. These early studies led to further analyses of three-way tables, which include estimating the common odds ratio, testing zero interaction and testing no association across strata, for