Motion Compensation based Multiple Inter Frame Interpolation Jeffin Gracewell .J Research Scholar Department of Electronics Engineering, Madras Institute of Technology, Chennai, India jgracewell@gmail.com Mala John Professor Department of Electronics Engineering, Madras Institute of Technology, Chennai, India malajohnmit@gmail.com Abstract — In this paper, a fast forward motion estimation (FFME) algorithm based on motion compensated frame rate up-conversion (MC-FRUC) with reduced time complexity is proposed. The proposed algorithm exploits the spatial correlation of the displacement vectors of the neighboring blocks in a frame. The algorithm achieves a reduction in the computational complexity by 85% compared to the existing single frame interpolation scheme, while maintaining almost the same objective quality. A multiple interpolation technique is also presented based on the proposed FFME technique. The experimental results demonstrate the suitability of the method for frame rate up conversion. Keywords— Motion estimaton; Motion Compensated Frame Interpolation; frame rate up-conversion. I. INTRODUCTION The technique used to reconstruct a video of higher frame rate from the one of lower frame rate by way of inserting interpolated frames periodically is referred to as MC-FRUC (motion compensated frame rate up conversion). FRUC can be used in applications like video format conversion, reduction of motion blur in liquid crystal display (LCD), slow motion replay and low bit-rate video communication [1]–[5]. Many surveillance cameras capture video with relatively low frame rate in order to adapt to the constraints on the bandwidth of the video surveillance network and memory requirements, necessitating frame interpolation at the display end. For video surveillance applications, when the frame rate is high, frame rate could be reduced before transmission in order to conserve bandwidth and frame interpolation techniques could be used at the display end. There are different ways of carrying out Frame Rate Up Conversion. Frame repetition is a primitive technique, wherein, the same reference frame is being repeated for interpolation. Frame averaging is another technique, wherein, the new frame is interpolated by averaging two consecutive frames. These algorithms are easier to implement but cause blurring, motion jerks and perceptible synthetic effects and MC-FRUC can substantially reduce these effects in the interpolated video. II. RELATED WORKS Motion Estimation (ME), Motion Analysis (MA) and Motion Compensated Interpolation (MCI) are the major steps in MC-FRUC. The objective of using motion vector (MV) in FRUC algorithm is not to find the minimum residual error but to find the true motion trajectory between two frames. ME is the primary step which helps to maintain the quality of the video signal in FRUC algorithm. The ME for MC-FRUC is mainly categorized into Unidirectional Motion Estimation (UME) and Bidirectional Motion Estimation (BME) approaches. Unidirectional ME suffers from holes and overlap in the occluded region of the interpolated frame. Holes occurs because of the absence of MV in that region and overlap occurs because of the presence of multiple motion vector in the interpolated frame. Several methods based on UME have been proposed in order to overcome holes and overlapped region in the interpolated frame. U.S. Kim et al [6] have proposed an intrapredicted hole interpolation (IPHI) method for interpolating the holes, thereby improving the quality of the image. D. Wang et al [7] have proposed two algorithms, viz., Block wise Directional Hole Interpolation BDHI and Irregular grid Expanded-Block Weighted Motion Compensation (IEWMC). C.Wang et al [8] have proposed a trilateration filtering method for reducing the interpolation error and to fill the holes. Spatial interpolation and Image inpainting have also been used to eliminate holes in the interpolated frames [9],[10]. However, BME is being preferred over UME because of the advantage that, it eliminates the holes and overlap in the interpolated frame [10]- [13]. The limitation of BME lies in the inaccuracy of MV when there is no temporal symmetry in motion of object between the previous and future frames with respect to the interpolated block. To overcome the limitation of the above two methods, a new method which is the combination of both the UME and BME known as Unidirectional and Bidirectional Motion Estimation, (UBME) is proposed. In UBME, UME is firstly used to generate the initial MVF, and then BME-based Motion Vector Refinement (MVR) algorithms are used for refining the MVs [14]-[16]. Tsai et al[15] have proposed particle based FRUC method which combines the hierarchical MV fields and particle based motion trajectory stage in estimating the MV, but suffers from high computational 261 978-1-5090-2684-5/16/$31.00 ©2016 IEEE