1010 Wang et al. / J Zhejiang Univ-Sci C (Comput & Electron) 2011 12(12):1010-1020 Journal of Zhejiang University-SCIENCE C (Computers & Electronics) ISSN 1869-1951 (Print); ISSN 1869-196X (Online) www.zju.edu.cn/jzus; www.springerlink.com E-mail: jzus@zju.edu.cn Robust optical flow estimation based on brightness correction fields ∗ Wei WANG †1 , Zhi-xun SU †1 , Jin-shan PAN 1 , Ye WANG 2 , Ri-ming SUN 1 ( 1 School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, China) ( 2 Department of Mathematics, Harbin Institute of Technology, Harbin 150006, China) † E-mail: garywangzi@gmail.com; zxsu@dlut.edu.cn Received Mar. 17, 2011; Revision accepted June 17, 2011; Crosschecked Nov. 4, 2011 Abstract: Optical flow estimation is still an important task in computer vision with many interesting applications. However, the results obtained by most of the optical flow techniques are affected by motion discontinuities or illumination changes. In this paper, we introduce a brightness correction field combined with a gradient constancy constraint to reduce the sensibility to brightness changes between images to be estimated. The advantage of this brightness correction field is its simplicity in terms of computational complexity and implementation. By analyzing the deficiencies of the traditional total variation regularization term in weakly textured areas, we also adopt a structure-adaptive regularization based on the robust Huber norm to preserve motion discontinuities. Finally, the proposed energy functional is minimized by solving its corresponding Euler-Lagrange equation in a more effective multi-resolution scheme, which integrates the twice downsampling strategy with a support-weight median filter. Numerous experiments show that our method is more effective and produces more accurate results for optical flow estimation. Key words: Optical flow field, Variational methods, Brightness correction fields, Median filter, Multi-resolution doi: 10.1631/jzus.C1100062 Document code: A CLC number: TP391.4 1 Introduction Estimating a displacement field for consecutive images from an image sequence is one of the major challenges in computer vision. It arises whenever one aims to identify correspondences between points in pairs of images. Examples include motion estima- tion (Gilland et al., 2008), tracking (Dessauer and Dua, 2010), and medical multi-modal registration (Wang et al., 2007). The resulting dense correspon- dence between pairs of points in either image can subsequently be used for the structure-from-motion algorithm (Fakih and Zelek, 2008), object recogni- tion (Efros et al., 2003), and other higher level tasks. * Project supported by the National Natural Science Foundation of China (No. U0935004) and an IDeA Network of Biomedical Research Excellence (INBRE) grant from the National Institutes of Health (NIH) (No. 5P20RR01647206) c Zhejiang University and Springer-Verlag Berlin Heidelberg 2011 Starting with the seminal work of Horn and Schunck (1981), most of state-of-the-art methods are based on minimizing the energy E(U )= E d (I 1 , I 2 , U )+ λE s (U ), (1) where the unknown optical flow field U : Ω → R 2 , and two input images I 1 and I 2 are both defined on the image domain Ω ⊂ R 2 . The data term E d (I 1 , I 2 , U ) measures the similarity of the input images for a given optical flow field, and the regular- ization term E s (U ) allows to impose some prior on the optical flow field U . Although a lot of work has focused on improving energy models and optimizing algorithms, there is still room for improvement. For the data term, the original energy model of optical flow relies on the brightness constancy con- straint. Any changes of the illumination in the scene violating the brightness constancy assumption lead