Covariance Estimation for SAD Block Matching Johan Skoglund and Michael Felsberg Computer Vision Laboratory Department of Electrical Engineering Link¨ oping University SE-581 83 Link¨oping, Sweden {skoglund,mfe}@isy.liu.se Abstract. The estimation of a patch position in an image is a long established but still relevant topic with many applications, e.g. pose es- timation and tracking in image sequences. In most systems the position estimate needs to be fused with other estimates, and hence, covariance information is required to weight the different estimates in the right way. In this paper we address the issue with covariance estimation in the case of sum of absolute difference (SAD) block matching. First, we derive the theory for covariance estimation in the case of SAD matching. Second, we evaluate the suggested method in a virtual 3D patch tracking scenario in order to verify the performance in real-world scenarios. 1 Introduction Motion information from images is useful for many applications. In this paper we focus on one type of method for doing this, called block matching. Typically a patch is given and the algorithm finds the best match in an image. A common problem is merging of different measurements. Most common is to average a number of similar measurements to reduce average error but the measurements might also come from different sensors. Optimal merging of the measurements, requires knowledge about the accuracy for each measurement. Accuracy of these measurements and information whether the error in the mea- surements are dependent or not is stored in the ”covariance matrix”. In this paper we present both the theory and also an evaluation of a method for estimating the covariance for block matching using sum of absolute difference (SAD), which is to our knowledge not yet an established topic. 1.1 Block Matching Block matching is a common name for all algorithms that try to find the position of a patch p, in an image b. This can be done in a number of different ways but the goal is to find the minimum of an error function min γ e(p, T (b, γ )) . (1) B.K. Ersbøll and K.S. Pedersen (Eds.): SCIA 2007, LNCS 4522, pp. 374–382, 2007. c  Springer-Verlag Berlin Heidelberg 2007