1982 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 43, NO. 6, NOVEMBER 1997 [17] E. Arıkan, “Lower bounds to moments of list size,” in IEEE Int. Symp. on Information Theory (Abstracts of Papers), (San Diego, CA, Jan. 1990), pp. 145–146. [18] P. Billingsley, Probability and Measure, 2nd ed. New York: Wiley, 1986. A Non-Shannon-Type Conditional Inequality of Information Quantities Zhen Zhang, Senior Member, IEEE, and Raymond W. Yeung, Senior Member, IEEE Abstract— Given discrete random variables , associated with any subset of , there is a joint entropy where . This can be viewed as a function defined on taking values in . We call this function the entropy function of . The nonnegativity of the joint entropies implies that this function is nonnegative; the nonnegativity of the conditional joint entropies implies that this function is nondecreasing; and the nonnegativity of the conditional mutual informations implies that this function is two-alternative. These properties are the so-called basic information inequalities of Shannon’s information measures. An entropy function can be viewed as a -dimensional vector where the coordinates are indexed by the subsets of the ground set . As introduced in [4], stands for the cone in consisting of all vectors which have all these properties. Let be the set of all - dimensional vectors which correspond to the entropy functions of some sets of discrete random variables. A fundamental information-theoretic problem is whether or not Here stands for the closure of the set . In this correspondence, we show that is a convex cone, , , but . For four random variables, we have discovered a conditional inequality which is not implied by the basic information inequalities of the same set of random variables. This lends an evidence to the plausible conjecture that for . Index Terms— Entropy, -Measure, information inequalities, mutual information. I. INTRODUCTION AND SUMMARY Let be jointly distributed discrete random variables with finite entropies. The basic Shannon’s infor- mation measures associated with these random variables include all joint entropies, all conditional entropies, all mutual informations, and all conditional mutual informations involving some of these random variables. For any subset of let (1) Let , where is the empty set, be a random variable taking a fixed value with probability . Define (2) Manuscript received October 30, 1995; revised February 15, 1997. This work was supported in part by the National Science Foundation under Grant NCR-9508282. Z. Zhang is with the Communication Sciences Institute, Department of Electrical Engineering Systems, University of Southern California, Los An- geles, CA 90089-2565. R. W. Yeung is with the Department of Information Engineering, The Chinese University of Hong Kong, NT, Hong Kong. Publisher Item Identifier S 0018-9448(97)07295-7. We see that when (3) which is the conditional entropy; when (4) which is the unconditional mutual information, and when and (5) which is the joint entropy. This means that the function covers all the basic Shannon’s information measures. In this corre- spondence, all logarithms are in base . It is well known that Shannon’s information measures satisfy the following inequalities. Proposition 1: For any three subsets , , and of , any set of jointly distributed random variables , with finite entropies (6) These inequalities are called the basic inequalities of Shannon’s information measures [4]. Let be the joint entropy function. For any set of jointly distributed random variables , the associated entropies can be viewed as a function defined on (7) The goal of this correspondence is to study this function for all possible sets of random variables with finite entropies. All basic Shannon’s information measures can be expressed as linear functions of the joint entropies. Actually, we have (8) The basic inequalities can be interpreted as a set of inequalities for the entropy function as follows. Proposition 2: For any set of jointly distributed random vari- ables , with finite entropies, the entropy function associated with these random variables has the following properties. 1) For any two subsets and of (9) Functions having this property are called two-alternative func- tions. 2) implies (10) Functions satisfying this property are called monotone nonde- creasing, and (11) It is easily seen from (8) that the first property corresponds to the nonnegativity of all mutual informations and condition mutual informations, and the second and third property correspond to the nonnegativity of all entropies and conditional entropies. 0018–9448/97$10.00  1997 IEEE