Motion Vector Processing Using Bidirectional Frame Difference in Motion Compensated Frame Interpolation Ai-Mei Huang and Truong Nguyen Video Processing lab ECE Dept, UCSD, La Jolla, CA 92093 E-mail: aihuang@ucsd.edu, nguyent@ece.ucsd.edu Abstract In this paper, we address the potential issues in bidi- rectional motion compensated frame interpolation when the received motion vector field is directly used. Based on this motion vector analysis, we therefore propose us- ing bidirectional motion vector processing method to elim- inate the ghost artifacts around the moving objects. The artifacts caused by large motion vector magnitude can be effectively removed by correcting motion vectors bidirec- tionally. Moreover, since the proposed motion vector se- lection process chooses the best motion vector from adja- cent motion vectors, the computational complexity can be greatly reduced comparing to motion estimation. We also demonstrate how to obtain clearer object edges by allow- ing displacement adjustment during the bidirectional mo- tion vector processing. Experimental results show that the proposed algorithm outperforms other methods in terms of visual quality. 1. Introduction To achieve real-time video display and meet the band- width requirement, video applications such as video tele- phony often subsample video frames temporally in the low bitrate encoding. As a result, we can often see the jerkiness artifacts when the video contains fast motion. In order to improve the temporal quality, motion-compensated frame interpolation (MCFI), which assumes the object moves along the motion trajectory and can be interpolated in the skipped frame, have been widely discussed. MCFI us- ing the block-based motion compensation can always pro- duce holes and overlapped areas in the interpolated frame due to no motion trajectory passing and multiple motion trajectories passing, respectively. Hence, computationally complex spatial interpolation is often used to recover hole regions [1]. Alternatively, bidirectional interpolation that obtains both forward and backward motions directly from the received motion vector field (MVF) for the interpolated frame can achieve low complexity MCFI. However, the received MVF is not always reliable and may contain intra-coded macroblocks (MBs) that do not have motion at all. This is because the encoder estimates motion vectors (MVs) based on maximizing the coding ef- ficiency instead of finding true motion. Therefore, we can often observe visual artifacts if the received MVs are di- rectly employed for MCFI. Many conventional methods aim to obtain true motion by re-estimating the MVs at the decoder. Ha proposed a overlapped block-based motion es- timation based on the assumption that larger block size mo- tion estimation provides more accurate motion than smaller block size [3]. To minimize the blockiness artifact for the interpolated frame, Lee adopted forward block matching algorithm first, and refined the motion compensations by weighting candidate predictions from different MVs that pass through the current interpolated block along the motion trajectory [5]. Instead of performing unidirectional motion estimation only, the work in [2] took MVs obtained from forward motion estimation as initial points and refine these MVs bidirectionally for the interpolated frame. To reduce the complexity of motion re-estimation, Zhai classified the received MVF first and performed bidirectional overlapped motion estimation for those unreliable MVs [7]. In addition to the motion re-estimation, MV correction for the received MVF is presented in [6] by adaptively averaging the center MV and surrounding MVs according to their corresponding coding difficulties. A thorough MV reliability analysis for the received MVF is presented in [4]. Those classified un- reliable MVs are further corrected by a proposed similarity constrained vector median filter. Exhaustive motion re-estimation methods may limit their applications on the mobile devices. Although MV process- ing methods have much less complexity, MV processing by averaging neighboring MVs or using vector median filter may not perform well once there is no reliable MV avail- able in the neighborhood. Therefore, we propose a MV pro-