Side information estimation and new symmetric schemes for multi-view distributed video coding q Thomas Maugey * , Béatrice Pesquet-Popescu Signal and Image Processing Department, Ecole Nationale Supérieure des Télécommunications (ENST), 46 rue Barrault, 75634 Paris Cedex 13, France article info Article history: Received 30 November 2007 Accepted 8 September 2008 Available online 19 September 2008 Keywords: Multi-view Distributed video coding Side information Fusion Rate-distortion analysis Wyner-Ziv Motion interpolation Long-term estimation abstract This paper deals with distributed video coding (DVC) for multi-view sequences. DVC of multi-view sequences is a recent field of research, with huge potential impact in applications such as videosurveil- lance, real-time event streaming from multiple cameras, and, in general, immersive communications. It raises however several problems, and in this paper we tackle two of them. Based on the principles of Wyner–Ziv (WZ) coding, in multi-view DVC many estimations can be generated in order to create the side information (SI) at the decoder. It has been shown that the quality of the SI strongly influences the global coding performances. Therefore, this paper proposes to study the contribution of multiple SI estimations (in the temporal and view directions) to the global performances. Moreover, we propose new symmetric schemes for longer group of pictures (GOP) in multi-view DVC and show that we can fur- ther exploit the long-term correlations using a new kind of estimation, called diagonal. For such schemes, several decoding strategies may be envisaged. We perform a theoretical study of the temporal and inter- view dependencies, and confirm by experiments the conclusion about the best decoding strategy. Ó 2008 Elsevier Inc. All rights reserved. 1. Introduction A new paradigm in video coding is the distributed video coding (DVC) which allows to move the computation complexity from the encoder to the decoder. This can be interesting in many applica- tions, whenever a compression is due to be done with a light hard- ware. This new coding paradigm is based on two results of information theory appeared in the 70’s [1,2], which show that with two correlated sources encoded independently and decoded jointly one can achieve the same performances as with the two sources encoded and decoded jointly. Therefore, the correlation between the frames is not exploited anymore at the encoder but only at the decoder. Theoretically, the performances of such a method can attain the ones of classical hybrid codecs (H.263, MPEG-4, H.264, etc.). In practice, for the moment, the perfor- mances are still below even though, in most cases, DVC performs better than Intra coding [3–5]. In this paper, we propose to study the distributed video coding of multi-view sequences. This recent field of research is very interesting for a wide range of applications, such as videosurveillance, real-time event streaming from multiple cameras, and immersive communications in general. We propose to tackle two of the numerous problems raised by DVC of multi- view sequences. In Wyner–Ziv coding, one or several estimations are generated at the decoder in order to build the side information (SI). It is proven that the general coding performances highly de- pend on the quality of this SI. We thus propose to study the contri- bution of multiple existing estimations for building the SI, and propose a new kind of estimation exploiting further dependencies. Moreover, we propose new symmetric schemes, in which many decoding strategies may be conceivable. We thus perform a theo- retical study based on temporal and inter-view dependencies in or- der to propose the best decoding strategy. After presenting a short summary of the theoretical distributed source coding framework in Section 2, we expose the main features of DVC implementation, the problems and their existing answers in Section 3. In Section 4, we introduce a new model for rate-distor- tion analysis of video sequences. In Section 5, we analyze the ideal estimation achieved by side information in a multi-view setting and we then propose new symmetric schemes with longer GOPs, based on the previous proposed model. This is followed by exper- imental results in Section 6, and finally the conclusion and future work are drawn in Section 7. 2. Theoretical results 2.1. Slepian and Wolf In 1973, Slepian and Wolf [1] studied the performances of the transmission of two correlated sources X and Y in many cases, 1047-3203/$ - see front matter Ó 2008 Elsevier Inc. All rights reserved. doi:10.1016/j.jvcir.2008.09.002 q Part of this work has been funded by the French ANR project no. ANR-FI-071215- 01-01 (ESSOR). * Corresponding author. E-mail addresses: maugey@telecom-paristech.fr (T. Maugey), pesquet@telecom- paristech.fr (B. Pesquet-Popescu). J. Vis. Commun. Image R. 19 (2008) 589–599 Contents lists available at ScienceDirect J. Vis. Commun. Image R. journal homepage: www.elsevier.com/locate/jvci