GENERATING H.264/AVC COMPLIANT BITSTREAMS FOR LIGHTWEIGHT DECODING OPERATION SUITABLE FOR MOBILE MULTIMEDIA SYSTEMS Kemal Ugur 1 , Jani Lainema 1 , Antti Hallapuro 1 , Moncef Gabbouj 2 1 Nokia Research Center, Tampere, Finland 2 Tampere University of Technology, Tampere, Finland ABSTRACT In this work, we propose novel encoder algorithms for the state-of-the-art video coding standard H.264, to generate decoder friendly video bitstreams. Using the proposed algo- rithms, it is possible to generate bitstreams requiring signifi- cantly less decoding complexity, with negligible effect on picture quality. This is achieved by using novel algorithms for mode decision and motion estimation that bias easy-to- decode motion vectors in a Rate-Distortion optimized fash- ion. Experimental results show that, more than 15% decod- ing complexity reduction is achieved with less than a 0.1 dB penalty on the average video quality. We believe that this approach has potential in various use cases especially in mobile multimedia systems, where the video decoder opera- tion is often dominating the handsets power consumption. 1. INTRODUCTION H.264/AVC is the state-of-the art video coding standard that is jointly developed by ISO/MPEG and ITU-T/VCEG study groups. When compared to the earlier video coding stan- dards, H.264/AVC achieves significantly better video quality at similar bitrates. Due to its high compression efficiency and network friendly design, H.264/AVC is gaining momen- tum in industry ranging from third generation mobile mul- timedia services, digital video broadcasting to handheld (DVB-H) to high definition digital versatile discs (HD- DVD). H.264 achieves increased compression efficiency with the expense of increased complexity for both the en- coders and the decoders. Similar to previous standards, the complexity of the H.264 encoder is typically much higher than that of the decoder. There exist several tools that sig- nificantly reduce the encoding complexity by resulting slightly lower video quality. These tools include for example fast motion estimation [1], fast mode decision [2] and dis- abling the use of some of the motion modes etc. The com- monality of all these is, they achieve complexity reduction by performing a complexity-video quality trade-off. For decoders, the situation is completely different. Since the decoder is strictly defined by the standard, it is not pos- sible to have lower complexity decoding by performing a similar video quality-decoding complexity trade-off. Even though the decoder has less complexity than the encoder, the importance of low complexity decoding is equally important as low complexity encoding, if not more. The reason for that is, in many applications such as DVD players, digital TV receivers etc, the end-user equipment has only the decoder implemented and the decoder block is the only codec related functional block adding complexity to the system. Even if the encoder and decoder co-exist in a system, such as in a video conferencing application, the decoding complexity could become very important if the encoder and decoder are running on different hardware platforms. In this work, we propose several encoding algorithms that can be used to generate fully H.264 compliant low- complexity bitstreams, which require significantly less de- coding complexity than bitstreams created with traditional encoding algorithms. More specifically, the focus of this work is to generate bitstreams that would require less amount of half-pixel and quarter pixel interpolations at the decoder, as interpolation step consumes most of the decod- ing processor cycles [3]. This is achieved in two stages. At the motion estimation stage, the candidate motion vectors having less decoding complexity are biased using a Lagran- gian based cost function. At the mode decision stage, the decoding complexity of each mode is estimated and the modes with less decoding complexity are favoured using a similar cost function. Using the proposed methods, the en- coder can generate low-complexity bitstreams having over 15% less decoding complexity with less than a 0.1 dB pen- alty on the video quality on average. This paper is organized as follows; Section 2 provides a brief analysis of the H.264 decoder complexity and the in- terpolation scheme. Section 3 presents the proposed motion estimation and mode decision algorithms to generate the low complexity bitstreams. Section 4 presents the simulation environment and the experimental results. Conclusions and discussions are presented in Section 5. 2. H.264 DECODER COMPLEXITY ANALYSIS Previous analyses on H.264 decoder complexity show that the motion compensation is the most computationally complex step at the decoder, followed by the deblocking filter process [3][4]. The high complexity in motion com- pensation is due to interpolation needed to decode motion vectors with half or quarter pixel accuracy. It was shown that, for a baseline H.264 decoder, this interpolation step takes around 39% of the execution time on average, and it can go up to 44% for some sequences. In the next subsec- tion, we first analyze the details of the H.264’s interpolation scheme and present source of its complexity. 2.1 Half-pixel and Quarter-pixel Interpolation H.264 allows usage of motion vectors with quarter and half pel precision to increase the accuracy of the motion predic- tion. For the case of integer motion vectors, the prediction signal contains the original values of the reference picture; otherwise the values at non-integer positions need to be in- II  33 142440469X/06/$20.00 ©2006 IEEE ICASSP 2006