OPTIMAL RATE CONTROL METHODS FOR FINE GRANULARITY SCALABLE VIDEO Sharadha Parthasarathy, Hayder Radha (parthas3, radha)@egr.msu.edu Department of Electrical and Computer Engineering Michigan State University, East Lansing MI, 48824. ABSTRACT In this paper we present two optimum (in a rate-distortion – RD – sense) rate-control algorithms for FGS scalable video. The proposed methods, which could compliment recently proposed RD FGS algorithms for bit allocation among different FGS frames, are targeted for optimum rate allocation among the different macroblock bitplanes within each FGS frame. The proposed methods are also designed to be very low in complexity to enable real-time streaming and fast off-line processing. Simulations with a variety of video sequences have been conducted and some of these simulations are presented in this paper. Based on these results, the proposed RD based rate- control algorithms provide a gain of 0.25 – 0.6 dB gain when compared with the traditional raster-scan bitplane FGS rate- control algorithm. 1. INTRODUCTION A primary concern in the transmission of video content over a network with varying bandwidth characteristics, e.g., the Internet, is inconsistent picture quality. Consequently, several optimal bit allocation schemes based on rate-distortion (RD) theory have been proposed for non-scalable and scalable video[1]-[5]. For example, [3] and [4] propose optimal RD based rate-allocation under dynamic transmission bandwidth conditions for the enhancement layer (EL) of MPEG-4 FGS (fine-granular scalability) coded video. These methods guarantee constant picture quality over a range of frames. However, once the optimal bitrate is determined for any frame, the FGS enhancement layer is traditionally being constrained to that bitrate using a raster-scan order rate-control mechanism. In this paper, we propose two new rate-control algorithms for the FGS enhancement layer. We show that implementing these algorithms to reach the target bitrate improves the rate-control performance curve in an optimum rate- distortion sense. We call these algorithms the Bitplane Rate Control (BRC) and the Cross-Bitplane Rate Control (CBRC) algorithms. The BRC and the CBRC algorithms are developed in the transform domain. The algorithms operate on a per-frame basis only. This flexibility of per-frame optimization can be achieved in FGS due to the absence of prediction among frames in the enhancement layer (MPEG-4 SNR FGS prediction is always based on the base-layer). Further, the proposed algorithms are designed to minimally increase the computational complexity over the existing raster-scan rate-control method. In addition, both BRC and CBRC are independent of the rate- control scheme used for the base-layer. We show that applying BRC or CBRC results in a 0.25-0.6dB improvement in performance of the rate-control curve in the transform domain. The remainder of this paper is organized as follows. Section 2 provides relevant background on the MPEG-4 FGS enhancement layer coding technique. Section 3 describes the optimization problem addressed by the BRC and CBRC algorithms. Sections 4 and 5 detail, respectively, the description and performance analysis of each of the two algorithms. Section 6 concludes the paper. 2. BACKGROUND The FGS enhancement layer is formed by DCT coding the difference in intensity between every pixel in the original frame and the corresponding pixel in the reconstructed base-layer frame[6]-[8]. This layer is coded using the bitplane coding method ([6],[9],[10]). The number of bitplane levels needed to represent the enhancement layer of a particular frame is given by, ( ) 2 max log | | 1 BP N C   = +   (1) Here, N BP is the number of bitplanes and |C| max is the maximum DCT magnitude of the given residual frame. The encoded enhancement layer bitstream is sent to a streaming server, which effectively performs a raster scan of the enhancement layer bitplanes. The scanning starts from the first macroblock (MB) through to the last on a particular bitplane. The first bitplane to be scanned for a given frame is the one that corresponds to the Most-Significant-Bitplane (MSB) position of the largest DCT coefficient in that frame. The server stops sending more enhancement layer information whenever the target available bitrate is reached for that frame. This is a coarse brute-force rate-control mechanism that does not meet the objective of optimal rate-control performance. In this paper, we setup the optimization problem to find a more efficient rate- control performance curve. We then provide a rate-distortion based solution in the context of the FGS enhancement layer. 3. THE OPTIMIZATION PROBLEM In this section, we identify the optimization problem for efficient rate-control for the FGS enhancement layer. Suppose M is the set of all macroblocks in the enhancement layer bitstream. Let µ be the set of macroblocks pruned by the streaming server during a simple raster-scan rate-control (Figure 1 (a)). It is to be noted that µ is a subset of M. Also, the macroblocks are of varying