[ standards in a NUTSHELL ] Heiko Schwarz and Mathias Wien The Scalable Video Coding Extension of the H.264/AVC Standard T he Scalable Video Coding extension (SVC) of the H.264/MPEG-4 Advanced Video Coding (AVC) standard (H.264/AVC) is the latest amendment for this successful specifica- tion. SVC allows partial transmission and decoding of a bit stream. The resulting (decoded) video has lower temporal or spatial resolution or reduced fidelity while retaining a reconstruction quality that is close to that achieved using the existing single-layer H.264/AVC design with the same quantity of data as in the partial bit stream. SVC provides net- work-friendly scalability at a bit stream level with a moderate increase in decoder complexity relative to single- layer H.264/AVC. Furthermore, it pro- vides the functionality of lossless rewriting of fidelity-scalable SVC bit streams to single-layer H.264/AVC bit streams. The SVC extension of H.264/AVC is suitable for video conferencing as well as for mobile to high-definition broadcast and professional editing applications. The H.264/AVC standard was present- ed in this column in the March 2007 issue of IEEE Signal Processing Magazine (pp. 148–153). The current article focuses on the SVC extension in terms of technology, performance, and targeted application scenarios. BACKGROUND International video coding standards such as H.261, MPEG-1, MPEG-2 Video, H.263, MPEG-4 Visual, and H.264/AVC have played an important role in the suc- cess of digital video applications. They provide interoperability among products from different manufacturers while allowing a high flexibility for implemen- tations and optimizations in various application scenarios. The H.264/AVC specification represents the current state- of-the-art in video coding. Compared to prior video coding standards, it signifi- cantly reduces the bit rate necessary to represent a given level of perceptual qual- ity—a property also referred to as increase of the coding efficiency. The desire for SVC, which allows on- the-fly adaptation to certain application requirements such as display and processing capabilities of target devices, and varying transmission conditions, originates from the continuous evolution of receiving devices and the increasing usage of transmission systems that are characterized by a widely varying connec- tion quality. Scalability has already been present in the video coding standards MPEG-2 Video, H.263, and MPEG-4 Visual in the form of scalable profiles. However, the provision of scala- bility in terms of picture size and recon- struction quality in these standards comes with a considerable growth in decoder complexity and a significant reduction in coding efficiency (i.e., bit rate increase for a given level of recon- struction quality) as compared to the cor- responding nonscalable profiles. These drawbacks, which reduced the success of the scalable profiles of the former specifi- cations, are addressed by the new SVC amendment of the H.264/AVC standard. MOTIVATION Video coding today is used in a wide range of applications ranging from multi- media messaging, video telephony and video conferencing over mobile TV, and wireless and Internet video streaming to standard- and high-definition TV broad- casting. In particular, the Internet and wireless networks gain more and more importance for video applications. Video transmission in such systems is exposed to variable transmission conditions, which can be dealt with using scalabili- ty features. Furthermore, video content is delivered to a variety of decoding devices with heterogeneous display and computational capabilities. In these heterogeneous environments, flexible adaptation of once-encoded content is desirable, at the same time enabling interoperability of encoder and decoder products from different manufacturers. OBJECTIVES The objective in the development of SVC was to enable the encoding of a high-quality video bit stream that con- tains one or more subset bit streams that can themselves be decoded with a complexity and reconstruction quality similar to that achieved using the exist- ing H.264/AVC design with the same quantity of data as that in the subset bit stream. Since the original H.264/AVC specification already includes the basic features necessary to enable scalability in terms of frame rate, the main objec- tive was to add scalability in terms of picture size and reconstruction quality (fidelity). At the same time, the objec- tive was to allow for straightforward and very-low-complexity manipulation and adaptation of scalable bit streams. Overall, the objective was to ensure the benefit of the scalable coding scheme IEEE SIGNAL PROCESSING MAGAZINE [135] MARCH 2008 Digital Object Identifier 10.1109/MSP.2007.914712 1053-5888/08/$25.00©2008IEEE THIS ARTICLE FOCUSES ON THE SVC EXTENSION IN TERMS OF TECHNOLOGY, PERFORMANCE, AND TARGETED APPLICATION SCENARIOS.