JOINT SOURCE-CHANNEL CODING FOR SCALABLE VIDEO USING MODELS OF RATE-DISTORTION FUNCTIONS Lisimachos P. Kondi Dept. of Electrical Engineering University at Buffalo The State University of New York Buffalo, NY 14260 Aggelos K. Katsaggelos Dept. of Electrical and Computer Engineering Northwestern University Evanston, IL 60208 ABSTRACT A joint source-channel coding scheme for scalable video is de- veloped in this paper. An SNR scalable video coder is used and Unequal Error Protection (UEP) is allowed for each scalable layer. Our problem is to allocate the available bit rate across scalable layers and, within each layer, between source and channel cod- ing, while minimizing the end-to-end distortion of the received video sequence. The resulting optimization algorithm we propose utilizes universal rate-distortion characteristic plots. These plots show the contribution of each layer to the total distortion as a func- tion of the source rate of the layer and the residual bit error rate (the error rate that remains after the use of channel coding). Models for these plots are proposed in order to reduce the computational complexity of the solution. Experimental results demonstrate the effectiveness of the proposed approach. 1. INTRODUCTION During the past few years there has been an increasing interest in multimedia communications over different types of channels. In recent days a significant amount of research has been focused on multimedia transmission over wireless channels. This is a complex and challenging problem due to the multipath fading characteris- tics of the channel. Source coding is concerned with the efficient representation of a signal. While bit errors in the uncompressed signal can cause minimal distortion, in its compressed format a single bit error can lead to significantly larger errors. Hence for transmission over an error prone channel, it is imperative that channel coding be em- ployed to make the data more resilient to channel errors by in- creasing the redundancy. Traditionally, source and channel coding have been consid- ered independently. The reason behind this is Shannon’s impor- tant information-theoretic result establishing the principle of sep- arability [1]. It states that the design of source and channel cod- ing can be separated without any loss in optimality as long as the source coding produces a bit rate that can be carried by the chan- nel. This principle relies on the crucial assumption that the source and channel codes can be of arbitrary long lengths. In practical situations, due to limitations on the computational power and pro- cessing delays this assumption does not hold. It is then of benefit to consider the problem of source and channel coding jointly. Joint source-channel coding is an active research area. A review of joint source-channel coding for wireless channels can be found in [2]. Source Encoder Channel Encoder Interleaver Channel Deinterleaver Channel Decoder Source Decoder Modulation Detection In Out Fig. 1. Block diagram of a typical video transmission system. In a compressed video bit stream the various parts of the bit stream are not equally important to the quality of the decoded video sequence. Thus, instead of protecting them equally, it would be advantageous to protect the most important parts of the bit stream more than the less important parts. This is the idea of data partitioning and Unequal Error Protection (UEP). In this work we apply UEP to the layers of a scalable bit stream. The break-up of the bit stream into subsets of varying qual- ity using a scalable codec lends itself naturally to employing an unequal error protection scheme. The base layer is typically bet- ter protected than the enhancement layers. This allows for added degrees of freedom in selecting the rates that will minimize the overall distortion. In [3], the benefits of using scalability in an er- ror prone environment are shown by examining all the scalability modes supported by MPEG-2 in an ATM network. The basic block structure of the system we are considering in this paper is shown in Fig. 1. We begin with a scalable video bit stream that is channel coded using a specified channel rate. This channel coded information is then interleaved and modulated for transmission over the channel. At the receiver the information is demodulated and deinterleaved. This received channel data is then decoded using a channel decoder and finally sent to the source decoder. In this paper we extend our previously reported results ([4, 5, 6]) by using a model for the Universal Rate Distortion Char- acteristics. The model used is similar to the one proposed in [7]. The rest of the paper is organized as follows. In section 2 we describe the elements of the video transmission system, i.e., scal- able video coding (section 2.1), channel encoding (section 2.2), and wireless channel (section 2.3). In section 3, the joint source coding optimization algorithm is described. In section 4, exper- imental results are presented and in section 5, conclusions are drawn.