730 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 12, NO. 8, AUGUST 2002 An Efficient Streaming and Decoding Architecture for Stored FGS Video Yi-Shin Tung, Ja-Ling Wu, Po-Kang Hsiao, and Kan-Li Huang Abstract—Fine granularity scalability (FGS) is the latest video-coding tool provided in Amendment 2 of the MPEG-4 stan- dard. By taking advantage of bitplane coding of DCT residues, the compressed bitstream can be truncated at any location to support the finest rate scalability in the enhancement layer. However, both frame buffer scanning several times in bitplane decoding and frame duplication simultaneously for base and enhancement layer decoding make FGS difficult in its implementation. In this paper, we propose a corresponding pair of efficient streaming schedule and pipeline decoding architecture to deal with the prescribed problems. The design may be applied to the case of streaming stored FGS videos and benefit FGS-related applications. Index Terms—Bitplane coding, fine granularity scalability, pipeline processing, scheduling, timing. I. INTRODUCTION F INE granularity scalability (FGS) [1]–[3] is the latest video compression technique provided in Amendment 2 of the MPEG-4 standard [4], [5]. By coding bitplane residues of DCT coefficients [6], [7], FGS can support the finest data cutoff in the enhancement layer. It can also provide two extra functions: frequency weight and selective enhancement, which augments the possibility of coded picture-quality improvement of the en- coding process. FGS is believed to be an excellent technique suitable for ap- plying to the bandwidth-varying network (such as the Internet) or bandwidth-sharing environments [8]. For example, in the case of a modern set-top box, video on-demand (VOD) service [9] is provided over a quality of service (QoS) guaranteed network (such as xDSL), while Internet access or phone service can also be activated at the same time. This means that bandwidth is shared and the available quantity for VOD transmission varies with time. Fig. 1 depicts possible bit-rate assignments for some basic services in a certain period of time. In this situation, FGS results in good performance because it has the best ability to make use of the remaining bit rates for improving video quality at any instance. Fig. 2 presents a comparison of the abilities to adapt available bit rate to a given application for multiple-layer coding and FGS coding. From Fig. 2, it is clear that FGS can always make full use of the available bandwidth. However, several problems have to be faced during the FGS implementation. First, although less computation is one of the targets in the original FGS design, the limited speed of Manuscript received December 2001; revised April 10, 2002. This work was supported in part by the National Science Council, R.O.C., under Contact NSC 90-2622-E-002-008. The authors are with the Department of Computer Science and In- formation Engineering, National Taiwan University, Taipei, Taiwan 106, R.O.C. (e-mail: tung@cmlab.csie.ntu.edu.tw; wjl@cmlab.csie.ntu.edu.tw; pkshiao@cmlab.csie.ntu.edu.tw; shino@cmlab.csie.ntu.edu.tw). Publisher Item Identifier 10.1109/TCSVT.2002.800855. Fig. 1. Examples of traffic bandwidth sharing among several applications. There are three services activated simultaneously over the 1.5 Mbits/s ADSL line: (a) VOD; (b) phone service; and (c) Internet access. It is clear that the remainders of bit rate change with time. Fig. 2. Bit-rate adaptation to the multiple-layer coding and the FGS. Two configurations of multiple-layer coding are assumed: 1) 600 kbits/s for the base stream and each additional stream of enhancement layer occupies 300 kbits/s and 2) 600 kbits/s for the base stream and each additional stream of enhancement layer occupies 200 kbits/s. FGS base layer is also set to 600 kbits/s. Obviously, the enhancement layer stream for FGS can always adapt to the total bit rate. the bitplane decoding handicaps it in real-time applications. The penalty comes mainly from bitplane decoding’s need for accessing large frame buffer several times. Second, the enhancement data of the referable frames (like I or P) needs to be queued up until the moment that the corresponding decoded frames in the base layer no longer need to be referenced. This queuing process will enlarge the size of the decoding buffers required for storing the enhancement data. Third, additional operations of frame buffer duplication will be introduced if we want to perform base-layer prediction and reconstruct enhanced 1051-8215/02$17.00 © 2002 IEEE