SPATIAL NON-STATIONARY CORRELATION NOISE MODELING FOR WYNER-ZIV ERROR RESILIENCE VIDEO CODING Yongsheng Zhang, Hongkai Xiong, Li Song, Songyu Yu Dept. of Electronic Engineering, Shanghai Jiao Tong University, Shanghai 200240, P.R. China ABSTRACT * Most of the Wyner-Ziv (WZ) video coding schemes in lite- rature model the correlation noise (CN) between original frame and side information (SI) by a given distribution whose parameters are estimated in an offline process. In this paper, an online CN modeling algorithm is proposed to- wards a more practical WZ-based error resilient video cod- ing (WZ-ERVC). In ERVC scenario, the side-information is typically generated from the error concealed picture instead of bi-directional motion prediction. The proposed online CN modeling algorithm achieves the so-called classification gain by exploiting the spatially non-stationary characteris- tics of the motion field and texture. The CN between the source and error concealed SI is modeled by a Laplacian mixture model, where each mixture component represents the statistical distribution of prediction residuals and the mixing coefficients portray the motion vectors estimation error. Experimental results demonstrate significant perfor- mance gains both in rate and distortion versus the conven- tional Laplacian model. Index Terms— Wyner-Ziv coding, spatial non- stationary, correlation noise modeling 1. INTRODUCTION In hybrid video coding schemes, motion estimation and mo- tion compensation (ME/MC) is adopted to exploit temporal redundancy between successive frames. Although ME/MC achieves high compression efficiency, transmitting the en- coded video may suffer from error propagations and lead to the well-known drifting phenomenon. Recently, inspired by the natural error resilient property of Wyner-Ziv (WZ) cod- ing, WZ-based error resilient video coding (WZ-ERVC) schemes have been proposed in literature [1]. The schemes integrate a joint source-channel coding framework for video, and show significant RD performance gains over conven- tional error resilient schemes, e.g., Intra refresh (IR) and forward error correction (FEC) [2]. The coding efficiency of WZ coding depends critically on the capability to model the correlation noise (CN) be- * *The work has been partially supported by the NSFC grants No. 60632040, No. 60772099 and the National High Technology Re- search and Development Program of China (863 Program) (No. 2006AA01Z322). tween the original frame and side information (SI) [3]. Tra- ditionally, the CN is modeled by a given distribution whose parameters are offline estimated by assuming that both the source data and corresponding side-information are availa- ble at the encoder side or the decoder side. It is undesirable since: (1) the encoder cannot have the error pattern in transmission, especially for applications with large end-to- end delay; (2) the decoder side cannot have the source data, otherwise transmission errors would be perfectly eliminated. To solve the problem, Brites et. al. proposed an online CN modeling algorithm for conventional WZ video coding, based on bi-directional ME/MC SI generation [4]. For ERVC applications, this paper proposes an online correlation noise modeling algorithm where SI is typically generated from the error concealed picture. In fact, the pre- diction residual has been de-correlated with the original frame, so the major task of error concealment resort to esti- mating the lost motion vectors according to the coherence property of the motion field and the spatial smoothness of neighboring blocks. The CN between the source and error concealed SI is modeled by a Laplacian mixture model, where each mixture component represents the statistical distribution of prediction residuals and the mixing coeffi- cients portray the motion vectors estimation error. Essential- ly, the proposed CN model describes the spatially non- stationary characteristics and achieves the so-called classifi- cation gain [5]. The rest of this paper is organized as follows: Section 2 provides a brief summary of WZ-ERVC framework. The proposed online CN modeling algorithm is studied in Sec- tion 3. Experimental results validate the efficiency of the proposed algorithm in Section 4. Section 5 concludes this paper. 2. THE WZ-BASED ERROR RESILIENT VIDEO CODING A general architecture of WZ-ERVC is shown in Fig. 1. at the encoder side denotes the current frame through the conventional predictive encoder, e.g., MPEG or H.26x en- gine. To eliminate temporal error propagation, the wave- form of some P-frames is protected by WZ coding. At the decoder side, the MPEG/H.26x bit-stream is firstly decoded. If current frame is not contaminated by transmission error, the error concealment (EC) module and WZ decoding mod- ule will be skipped. Otherwise, the EC module would be 2929 978-1-4244-5654-3/09/$26.00 ©2009 IEEE ICIP 2009