IOSR Journal of Computer Engineering (IOSRJCE) ISSN: 2278-0661 Volume 2, Issue 3 (July-Aug. 2012), PP 12-24 www.iosrjournals.org www.iosrjournals.org 12 | Page Message Passing Algorithm: A Tutorial Review Kavitha Sunil, Poorna Jayaraj, K.P. Soman Centre for Excellence in Computational Engineering and Networking Amrita Vishwa Vidyapeetham, India-641112. Abstract: This tutorial paper reviews the basics of error correcting codes like linear block codes and LDPC. The error correcting codes which are also known as channel codes enable to recover the original message from the message that has been corrupted by the noisy channel. These block codes can be graphically represented by factor graphs. We mention the link between factor graphs, graphical models like Bayesian networks, channel coding and compressive sensing. In this paper, we discuss an iterative decoding algorithm called Message Passing Algorithm that operates in factor graph, and compute the marginal function associated with the global function of the variables. This global function is factorized into many simple local functions which are defined by parity check matrix of the code. We also discuss the role of Message Passing Algorithm in Compressive Sensing reconstruction of sparse signal. Index Terms—Linear block code, factor graph, LDPC, Bayesian networks, belief propogation, Message passing algorithm, sum product algorithm, Compressive sensing. I. Introduction This paper provides tutorial introduction to linear block codes, factor graph, Bayesian network and message passing algorithm. In coding theory, to enable reliable delivery of bit stream from its source to sink over noisy communication channel error correcting codes like linear block codes and LDPC are introduced. While the message is sent from source to sink, error is introduced by the noisy channel. Error correcting techniques help us to recover the original data from the distorted one. These error correcting codes are graphically represented using factor graphs and an iterative decoding algorithm for the same is developed. Message passing algorithm which is an iterative decoding algorithm factorizes the global function of many variables into product of simpler local functions, whose arguments are the subset of variables. In order to visualize this factorization we use factor graph. Here we discuss the message passing algorithm, called the sum product algorithm. This sum product algorithm operates in a factor graph and compute various marginal function associated with global function. Then we link factor graphs with graphical models like Bayesian (belief) networks. Bayesian networks show the factorization of joint distribution function (JDF) of several random variables. MacKay and Neal, was the first to connect Pearl‟s ‟Belief Propagation‟ algorithm with coding. In message passing algorithm the messages passed along the edges in the factor graph are probabilities or beliefs. In this paper we tried to unify the work [9], [10],[11],[5],[7],[17].In Section II, we review the fundamentals of binary linear block codes, factor graphs and LDPC. Graphical models like Bayesian networks are developed as a key to iterative decoding algorithm in Section III. In Section IV we discuss the message passing algorithm which decodes the original data from the distorted data. Section V discuss the probability domain version of sum product algorithm. Section VI discuss the role of Message Passing Algorithm in Compressive sensing reconstruction of sparse signal. Approximate Message Passing(AMP)algorithm for compressive sensing was recently developed by David L. Donoho,Arian Maleki and Andrea Montanaria in [17].In section VII we conclude. II. Binary Linear Block Codes Linear block codes are conceptually simple codes that are basically an extension of single-bit parity check codes for error detection. In channel encoder a block of k message bits is encoded into a block of n bits by adding ( ) n k  number of check bits. These ( ) n k  number of check bits are derived from k information bits. Clearly n k  and such a code is called (,) nk block code. A (,) nk block code is said to be a (,) nk linear block code if it satisfies the condition that any linear combination of codeword is still a codeword. A (,) nk binary linear block code is a finite set 2 n C F  of codewords x . Each codeword is binary with length n . C contains 2 k codewords. An (,) nk block code C is a mapping between a k -bit message vector u