M. Kim et al.: Spatial Error Concealment for H.264 Using Sequential Directional Interpolation Contributed Paper Manuscript received August 4, 2008 0098 3063/08/$20.00 © 2008 IEEE 1811 Spatial Error Concealment for H.264 Using Sequential Directional Interpolation Myounghoon Kim, Hoonjae Lee, and Sanghoon Sull Abstract — Error concealment at the decoder restores erroneous macroblocks (MBs) caused by channel errors. In this paper, we propose a novel spatial error concealment algorithm based on prediction modes of intra-blocks which are included in a H.264-coded stream and highly correlated to the direction of local edge within the block. The key contribution is to sequentially interpolate each pixel in a lost MB by utilizing edge directions and strengths efficiently estimated from the neighboring blocks, preserving local edge continuity for more visually acceptable images. The proposed scheme is simple to implement and more reliably recover high-detailed content in corrupted MBs. The experimental results shows the proposed method achieves reduction in speed by 14%~39% as compared to existing method, and outperforms them in PSNR by 0.5~1dB as well as in subjective visual evaluation 1 . Index Terms — Spatial error concealment, Intra prediction modes, Sequential Directional interpolation, Block-loss recovery, H.264. I. INTRODUCTION For video transmission over bandwidth-limited networks, digital video compression standards such as H.264/AVC [1] use a combination of various advanced features to improve the compression. However, compressed video streams are vulnerable to transmission errors due to both of packet losses and delays that are almost inevitable in video transmission over wireless channels as well as the Internet. Therefore, most video encoders adopt various methods to achieve error resilience for transmission over noisy channels so as to avoid or reduce the possible visual distortion. There have been numerous studies on error resilient compression and decompression methods in an attempt to achieve robust video transmission. One approach [2] is to use a feedback channel to request retransmission or adjust the encoding modes according to channel conditions. Another approaches [3]-[5] are to insert redundant information into compressed video streams, which are efficient in stopping 1 This work was supported in part by the Korea University Grant. Myounghoon Kim is with the Department of Electronics and Computer Engineering, Korea University, 1, 5-ka, Anam-dong, Sungbuk-ku, Seoul 136- 701, Korea. (Telephone: +82-2-3290-3805, e-mail: mhkim@mpeg.korea.ac.kr). Hoonjae Lee is with the Department of Electronics and Computer Engineering, Korea University, 1, 5-ka, Anam-dong, Sungbuk-ku, Seoul 136- 701, Korea. (Telephone: +82-2-3290-3805, e-mail: hoonjae@mpeg.korea.ac.kr). Sanghoon Sull is with the Department of Electronics and Computer Engineering, Korea University, 1, 5-ka, Anam-dong, Sungbuk-ku, Seoul 136- 701, Korea. (Telephone: +82-2-3290-3805, e-mail: sull@mpeg.korea.ac.kr). Corresponding Author: Sanghoon Sull error propagation but may not be acceptable in some interactive applications due to extra delays and more computation power required. A variety of spatial interpolation techniques for restoring an erroneous macroblock (MB) at the decoder side have been proposed, whose main objective is to estimate the lost information usually occurred due to transmission errors by utilizing the correctly received information. One of the typical spatial error concealment (SEC) methods [6], [7] is to interpolate each pixel in a lost MB from intact pixels in adjacent MBs. This linear interpolation scheme is a simple yet effective method for smooth images. More advanced approaches were proposed to adaptively recover the lost MB to improve quality of concealed frame. The algorithm [8] proposed the multi- directional interpolation (MDI) scheme which performs pixel domain interpolation along eight possible edge directions and considers the cases of both single edge and multiple edges. Kim et al proposed [9] a block loss recovery method based on fine directional interpolation (FDI), which extracts spatial direction vectors from the edge information of the neighboring images and then the spatial direction vectors are adaptively applied to interpolate lost pixels. These methods work well on smooth and regular blocks. However, a blurring effect is often seen in the concealed image if the lost MB contains low frequency components since the lost pixels are restored by an average of pixel values along multiple directions. Also they require a lot of computation on edge detection and reconstruction. Zeng and Liu [10] introduced the geometric-structure-based (GSB) SEC using a spatial directional interpolation scheme, which utilizes the local geometric information extracted from the surroundings. The two nearest surrounding layers of pixels of a missing block are converted into a binary pattern to reveal the local geometrical structure. The missing pixels are interpolated in a way to preserve the local geometrical structures. However, the directional interpolation could be sensitive due to the use of the angle information when the transition points are connected. Thus, the retrieved edges may not be faithful to the original ones. Park et al. [11] suggested the recovery of the image blocks using a method of alternating projections. Other approaches based on fuzzy logic reasoning [12], [13], are also widely used. They recover both the low and high frequency components by using a vague similarity relationship between a lost block and its neighboring blocks. However, the main drawback of these methods is that they have heavy computational load to be used for real-time applications due to its iterative procedures. Authorized licensed use limited to: Korea University. Downloaded on December 28, 2008 at 23:18 from IEEE Xplore. Restrictions apply.