Annals of Mathematics and Artificial Intelligence, 2 (1990) 367-382 367 INFORMATION AND PROBABILISTIC REASONING Wilson X. WEN Artifical Intelligence Systems, Telecom Research Laboratories, P.O. Box 279, 770 Blackburn Road, Clayton, Victoria 3168, Austrafia Abstract In this paper, the relationship between information and reasoning is investigated and a parallel reasoning method is proposed based on information theory, in particular the principle of minimum cross entropy. Some technical issues, such as multiple uncertain evidence, complicated constraints, small directed cycles and decomposition of underlying networks, are discussed. Some simple examples are also given to compare the method proposed here with other methods. Keywords: Information theory, information measure, entropy, probabilistic reasoning, rea- soning under uncertainty, Jeffrey's rule, Boltzmann-Jeffrey machine networks. 1. Introduction The uncertainty principle in physics formulated by Werner Heisenberg "sig- naled an end of Laplace's dream of a theory of science, a model of the universe that would be completely deterministic: one certainly cannot predict future events exactly if one cannot even measure the present state of the universe precisely!" [6]. In a world where almost nothing is certain, how certain we are about a conclusion from reasoning depends on how much information we have for the present state and the evidence observed. In this paper, we discuss the relationship between information and reasoning, and propose a parallel reasoning method based on information theory, in particular the principle of Minimum Cross Entropy (MCE). The next section introduces some basic concepts of information theory. Section 3 describes the principle of minimum cross entropy - the main theoretical basis of our method. Section 4 discusses a special kind of causal network - the recursive causal network. In section 5, a new parallel reasoning method - MCE reasoning in Boltzmann-Jeffrey machine networks - is introduced. Section 6 is dedicated to some more technical issues, such as parallelism of our method, multiple uncertain evidence, directed cycles in belief networks, decomposition of underlying networks, and complicated constraints. Some comparisons are also given between our method and some other methods in this section. Finally, section 7 gives our conclusions. 9 J.C. Baltzer A.G. Scientific Publishing Company