PROCEEDINGS OF THE IEEE, VOL. 95, NO. 6, JUNE 2007 DRAFT-1 Joint Source-Channel Turbo Techniques for Discrete-Valued Sources: from Theory to Practice Xavier Jaspar, Graduate Student Member, IEEE, Christine Guillemot, Senior Member, IEEE and Luc Vandendorpe, Fellow, IEEE Abstract— The principles which have been prevailing so far for designing communication systems rely on Shannon’s source and channel coding separation theorem [1]. This theorem states that source and channel optimum performance bounds can be approached as close as desired by designing independently the source and channel coding strategies. However, this theorem holds only under asymptotic conditions, where both codes are allowed infinite length and complexity. If the design of the system is constrained in terms of delay and complexity, if the sources are not stationary, or if the channels are non-ergodic, separate design and optimization of the source and channel coders can be largely suboptimal. For practical systems, joint source-channel (de)coding may reduce the end-to-end distortion. It is one of the aspects covered by the term cross-layer design, meaning a re-thinking of the layer separation principle. This article focuses on recent developments of joint source-channel turbo coding and decoding techniques, which are described in the framework of normal factor graphs. The scope is restricted to lossless compression and discrete-valued sources. The presented techniques can be applied to the quantized values of a lossy source codec but the quantizer itself and its impact are not considered. Index Terms— Channel code; cross-layer design; iterative de- coding; joint source-channel code; source code; turbo principle I. I NTRODUCTION E FFICIENT multimedia communication over time-varying mobile and/or wireless channels remains a challenging problem. A communication system is in general composed of a source coder, which maps the source symbols into a compressed binary representation and of a channel coder which maps the binary sequence into coded bits or waveforms for transmission. Similarly, the receiver is composed of a channel decoder and a source decoder. The source coder aims at reducing the bit rate to save bandwidth while the channel coder reintroduces some redundancy in order to protect the transmitted bits against potential channel noise. So, the two steps are quite antagonistic. Techniques where the source coder-decoder pair and the channel coder-decoder pair are designed and optimized independently are called tandem techniques [1]. The source code design in this case assumes that the error probability at the output of the channel decoder is Manuscript received August 25, 2006; revised February 26, 2007. This work was supported in part by the Federal Office for Scientific, Technical and Cultural Affairs, Belgium, under IAP contract No P5/11. The work of X. Jaspar is supported by the F.N.R.S., Belgium. X. Jaspar and L. Vandendorpe are with the Communications and Remote Sensing Laboratory, Universit´ e Catholique de Louvain, B-1348 Louvain-la- Neuve, Belgium (e-mail: jaspar@tele.ucl.ac.be; vandendorpe@tele.ucl.ac.be). C. Guillemot is with the IRISA-INRIA, F-35042 Rennes Cedex, France (e-mail: christine.guillemot@irisa.fr). zero. A non-zero error probability at the output of the channel decoder, which is unavoidable for finite-length channel codes, may have a strong effect on the end-to-end source distortion or source symbol error rate. Joint source-channel techniques, as the name suggests, try to optimize the transmission per- formance jointly, i.e., without a strict separation. Intuitively, since this optimization is a less constrained mathematical problem, its solution can only be better than or equal to tandem techniques. Joint source-channel techniques may hence reduce the source distortion and achieve better performance in the case of practical systems with constrained delay and complexity. This paper focuses on joint source-channel turbo tech- niques [2]–[24] and attempts to give a unified overview of this field. We have chosen the normal factor graphs [25] for the presentation of the different state models and decoding algo- rithms. These graphs are detailed in this special issue, in [26]. Together with the Sum-Product algorithm, they form [25]–[28] indeed a unifying framework to present turbo and turbo-like techniques. A factor graph is a graphical representation of a factorization, e.g., the factorization of the joint probability of all random variables involved in a transmission chain. As such, it is a powerful tool to analyze graphically the structure of stochastic dependencies and constraints between variables, as well as for reading out conditional independence relations in a model. In a sense, a factor graph is a substitute for equations and provides a clear overview of the transmission chain. As for the Sum-Product algorithm, its execution on factor graphs provides efficient (iterative) estimation algorithms. Actually, many decoding algorithms from the turbo/soft literature can be “re-discovered” with it and new ones can be developed. On Markov chains for example, it is equivalent to the BCJR algorithm [29], historically discovered before. The scope of the paper is restricted to discrete-valued sources. The presented techniques can still be applied to quantized values but the quantizer itself and its impact are not considered. Continuous-valued sources and quantizers in turbo systems are studied in [3], [8], [13], [22]. Considering initially no channel coder, several classical soft source estimation algorithms are presented, with the maximum a posteriori (MAP) and minimum mean square error (MMSE) criteria. These algorithms are described firstly with fixed length codes (FLCs) as a source code. FLCs make indeed the problem much simpler since the symbol boundaries in the noisy bit stream are known a priori. The general case of sources encoded with variable length codes (VLCs) and arithmetic codes (ACs) is then considered. With VLCs and