5th International Conference on Visual Information Engineering VIE 2008, Julho 29 – Agosto 1, Zi’an, China A NOVEL APPROACH TO JOINT VIDEO CODING Luis Teixeira*, Luís Corte-Real † * Research Center for Science and Technology in the Arts, Universidade Católica Portuguesa, Portugal, lteixeira@porto.ucp.pt † Faculdade de Engenharia da Universidade do Porto / INESC PORTO, Portugal, lreal@inescporto.pt Keywords: Subjective quality, structural information, joint video coding, H264 joint video coding. Abstract It is expected that future delivery of Digital TV signals will use H.264. This paper presents a novel coding scheme for multi-program video transmission in which the channel capacity is distributed among the programs according to the program complexities resulting in a more uniform overall image quality. A complexity bit rate control algorithm based on the Structural Similarity Index (SSIM) is proposed. SSIM metric is presented under the hypothesis that the Human Visual System (HSV) is very specialized in extracting structural information from a video sequence but not in extracting the errors. Thus, a measurement on structural distortion should give a better correlation to the subjective impression. Computer simulations have demonstrated very promising results showing joint coding is able to effectively control the complexity of the multi-program encoding process whilst improving overall subjective compared to independent coding and algorithms based on traditional distortion/quality metrics. 1 Introduction During recent years, international standardization bodies have developed several video coding standards such as MPEG-1, MPEG-2, MPEG-4, H.263 and H.264 [5]. These standards are based on a hybrid motion-compensated predictive video coding approach obtaining high compression ratios by removing both spatial and temporal redundancies existing in video sources. The H.264/MPEG-4 Advanced Video Coding standard (H.264/AVC) [5], also referred as ITU-T Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4 Part 10), is the latest video coding standard jointly developed by the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). H.264/AVC has accomplished a considerable progress regarding coding efficiency, substantially enhanced error robustness, and increased flexibility and scope of applicability relative to its predecessors [14]. It covers all frequent video applications ranging from mobile services and videoconferencing to IPTV, HDTV, and HD video storage [14]. In TV multi-programme broadcast systems, the use of H.264/AVC standards allows economies in transmission bandwidth of programs, while providing a service with a higher quality regarding current systems [15]. TV viewers receive video programs from a number of different video content providers via mixed transmission channels. The distribution of an equal fraction of the channel capacity which occurs when the multiple video programs are independent encoded will produce different picture quality among the video programs as the complexity varies from program and during the broadcast. One possible solution can be to allocate different bit rates to each video encoder based on the expected image complexity of the signal to be encoded. Such an allocation scheme should be dynamically adjusted over time, depending on the relative complexity of each channel. This process is called statistical multiplexing (stat-mux). In this paper, we propose a statistical multiplexing rate control scheme for H.264 coding of video sequences. To obtain uniform picture quality within multiple video programs, a novel complexity measure based on Structural Similarly Information is proposed, which can assess the coding complexity of the current frame regarding its perceptual quality. Furthermore, it can be implemented in each video encoder in a very simple way without resulting in an excessive increase in the computational power. In Section 2, Video Quality Assessment is introduced regarding objective and subjective video quality. The approaches are presented: SSIM and SAMVIQ. In Section 3, the proposed scheme is described and in Section 4 results are presented and discussed. 2 Video Quality Assessment Video programs are produced to be displayed to human viewers. As a result it is their opinion regarding the video quality that is relevant. Informal and formal subjective measurements have always been, and will continue be used to evaluate system performance from the design lab to the operational environment [8][3]. Although the need for an objective testing method of picture quality is clear, subjective testing is too complex and provides too much variability in results. However, since it is the observer’s opinion of picture quality that counts, any objective measurement system must have good correlation with subjective results for the same video system and test scenes. To be able to incorporate HVS model into broadcasting encoding system could result in additional improve of the coding efficiency and enhance video quality.