876 IEEE TRANSACTIONS ON MULTIMEDIA, VOL. 6, NO. 6, DECEMBER 2004 Vector Rational Interpolation Schemes for Erroneous Motion Field Estimation Applied to MPEG-2 Error Concealment Sofia Tsekeridou, Member, IEEE, Faouzi Alaya Cheikh, Moncef Gabbouj, Senior Member, IEEE, and Ioannis Pitas, Senior Member, IEEE Abstract—A study on the use of vector rational interpolation for the estimation of erroneously received motion fields of MPEG-2 predictively coded frames is undertaken in this paper, aiming further at error concealment (EC). Various rational interpolation schemes have been investigated, some of which are applied to different interpolation directions. One scheme additionally uses the boundary matching error and another one attempts to locate the direction of minimal/maximal change in the local motion field neighborhood. Another one further adopts bilinear interpolation principles, whereas a last one additionally exploits available coding mode information. The methods present temporal EC methods for predictively coded frames or frames for which motion information pre-exists in the video bitstream. Their main advantages are their capability to adapt their behavior with respect to neighboring mo- tion information, by switching from linear to nonlinear behavior, and their real-time implementation capabilities, enabling them for real-time decoding applications. They are easily embedded in the decoder model to achieve concealment along with decoding and avoid post-processing delays. Their performance proves to be sat- isfactory for packet error rates up to 2% and for video sequences with different content and motion characteristics and surpass that of other state-of-the-art temporal concealment methods that also attempt to estimate unavailable motion information and perform concealment afterwards. Index Terms—Error concealment, motion field estimation, MPEG-2 compression, temporal concealment, transmission er- rors, vector rational interpolation. I. INTRODUCTION T RANSMISSION of highly compressed video bitstreams (e.g., MPEG-2 video bitstreams) through physical com- munication channels is liable to errors, being either packet/cell loss or bit errors (isolated or in bursts). These types of errors are usually referred to as bitstream errors and they frequently result in loss of the decoder synchronization, which becomes unable to decode until the next resynchronization codeword in the re- ceived bitstream. Since resynchronization points are placed at Manuscript received October 3, 2001; revised January 7, 2003. This work was supported by the European RTN Project RTN1–1999-00177 – MOUMIR. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Heather Hong. S. Tsekeridou is with the Electrical and Computer Engineering Depart- ment, Democritus University of Thrace, 67100 Xanthi, Greece (e-mail: tsekerid@ee.duth.gr). F. A. Cheikh and M. Gabbouj are with the Institute of Signal Processing, Tampere University of Technology, FIN-33101 Tampere, Finland (e-mail: faouzi@cs.tut.fi; Moncef.Gabbouj@tut.fi). I. Pitas is with the Department of Informatics, University of Thessaloniki, Thessaloniki 54124, Greece (e-mail: pitas@zeus.csd.auth.gr). Digital Object Identifier 10.1109/TMM.2004.837266 the header of each slice by an MPEG-2 coder, unless otherwise specified, transmission errors may lead to partial or entire slice information loss (loss of prediction errors, coding modes, mo- tion vectors). Such loss of synchronization leads to observable deterioration of the decoded sequence quality, which, in the case of MPEG-2 coding, is further attributed to the use of VLC and differential coding. Furthermore, due to the use of motion-com- pensated prediction, temporal error propagation to predictively coded frames inside a group of pictures (GOP), i.e., propaga- tion errors, leads to even worse results. A number of methods have been proposed in the literature to achieve error resilience and thus deal with the above-mentioned problem. A nice overview of such methods is given in [1]. One of the ways to solve this problem is the implementation of error concealment (EC) methods at the decoder side. Such methods exploit the spatial and/or temporal correlations inside and among correctly received neighboring regions/frames [2]–[12]. A subclass of EC methods focuses on the “optimal” estimation of erroneously received motion fields based on available sur- rounding information. Among the methods belonging in this subclass, one may distinguish: • the zero motion EC (ZM EC), which sets lost motion vec- tors to zero and, thus, performs temporal replacement; • the motion-compensated EC (MC-AV or MC-VM EC), which calculates the average or vector median of adjacent predictively coded motion vectors; • the boundary matching algorithm EC (BMA EC), which defines a number of motion vector candidates and selects the optimal one that minimizes the boundary matching error [13]; • the motion vector estimation by boundary optimizing EC (MVE-BO EC), a fast suboptimal version which firstly evaluates the vector median (a MAP motion estimate) of adjacent predictively coded motion vectors, then defines search regions in reference frames around the block pointed at by the vector median, and finally looks for the optimal motion vector by minimizing the boundary matching error [2]; • the forward-backward block matching EC (F-B BM EC), which performs temporal block matching of adjacent blocks in order to estimate their “best match” in the reference frames by MAD minimization [10]; • the decoder motion-vector estimation EC (DMVE EC), a variant of F-B BM EC which, however, uses smaller 1520-9210/04$20.00 © 2004 IEEE