INTERNATIONAL JOURNAL OF COMMUNICATION SYSTEMS Int. J. Commun. Syst. 2005; 18:449–464 Published online 15 March 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/dac.711 Comparing the streaming of FGS encoded video at different aggregation levels: frame, GoP, and scene Philippe de Cuetos 1,z,} , Patrick Seeling 2,} , Martin Reisslein 2,n,y and Keith W. Ross 3,k 1 Institut Eurecom, Sophia-Antipolis, France 2 Department of Electrical Engineering, Arizona State University, Goldwater Center MC 5706, Tempe AZ 85287-5706, U.S.A. 3 Polytechnic University, 6 MetroTech Center, Brooklyn, NY 11201, U.S.A. SUMMARY Fine granularity scalability (FGS), a new coding technique that has recently been added to the MPEG-4 video coding standard, allows for the flexible scaling of each individual video frame at very fine granularity. This flexibility makes FGS video very well suited for rate-distortion optimized streaming mechanisms, which minimize the distortion (i.e. maximize the quality) of the streamed video by transmitting the optimal number of bits for each individual frame. The per-frame optimization of the transmission schedule, however, puts a significant computational burden on video servers and intermediate streaming gateways. In this paper we investigate the rate-distortion optimized streaming at different video frame aggregation levels. We find that compared to the optimization for each individual video frame, optimization at the level of video scenes reduces the computational effort dramatically, while reducing the video quality only very slightly. Copyright # 2005 John Wiley & Sons, Ltd. KEY WORDS: fine granularity scalability; multimedia communications; performance evaluation; rate- distortion optimized streaming; scalable video; video scene; video streaming 1. INTRODUCTION The MPEG-4 video coding standard has recently been augmented by fine granularity scalability (FGS), which has been designed to increase the flexibility of video streaming. An FGS encoded Contract/grant sponsor: National Science Foundation; contract/grant numbers: Career ANI-0133252; ANI-0136774 Contract/grant sponsor: State of Arizona; contract/grant number: IT301 Contract/grant sponsor: Sun Microsystems Received March 2004 Revised September 2004 Accepted October 2004 Copyright # 2005 John Wiley & Sons, Ltd. y E-mail: reisslein@asu.edu z E-mail: philippe.de-cuetos@enst.fr } He is now with ENST, Paris, France. } E-mail: patrick.seeling@asu.edu n Correspondence to: Martin Reisslein, Department of Electrical Engineering, Arizona State University, Goldwater Center MC 5706, Tempe AZ 85287-5706, U.S.A. k E-mail: ross@poly.edu