REGULARIZED ESTIMATION OF OCCLUDED DISPLACEMENT VECTOR FIELDS ABSTRACT Damon L. Tull Aggelos K. Katsaggeios Department of Electrical and Computer Engineering Northwestern University Evanston, Illinois 60208-3118 damon@eecs.nwu. edu, aggkfleecs.nwu.edu Occluded regions and motion boundaries introduce dis- placement vector field (DVF) discontinuities that must be reconciled to accurately estimate image flow. In this work, the robust regularized estimation of the DVF is considered in the presence of these discontinuities. A robust convex es- timation criterion is presented that preserves motion bound- aries and allows for a globally optimal estimate of the DVF. A new class of robust convex measures is introduced for edge preserving regularization and an occlusion weighted gradient is proposed as mechanism for managing DVF dis- continuities due to occlusion. Results using synthetic image sequences are presented. 1. INTRODUCTION The management, detection and estimation of discontinu- ous features in an image scene is vital to a wide variety of applications including image sequence processing (compres- sion, interpolation and filtering), machine vision and medi- cal imaging. Discontinuities convey information that can be used to estimate the structure (depth) and/or motion of an image scene. The effective management of discontinuities is essential for the accurate estimation of these features. Most traditional formulations attempt to find the hor- izontal and vertical components of the DVF, V(Z, j) and U(Z, j), respectively that minimize the displaced frame dif- ference (DFD), DFD(i, j) = p(i,j) - p-’(i - U(i,j), j - v(i,j)) = f:(i.m(i.o + f:(~jj)u(i~) + ff! (1) where jk (z, j), is the intensity of the pixel at location (z, j) in the kth frame and ~~, $$ and j$ are the spatial (hor- izontal and vertical) and temporal first derivatives of j~, respectively. The DFD is a non-linear measure of how well the DVF estimate accounts for differences in the frames of interest. The unconstrained global minimization of Eq. (1) is an ill-posed problem, requiring techniques such as regu- larization to obtain a unique (and meaningful) solution. A well-posed estimate of the DVF can be obtained from the minimization of the regularized DFD criterion, NM Q(u, r),A) = ~ ~ rJ(DFD(i, j)) +A)z: ;; [@([vu(i, j)l) + W7qi.i)l)] i=l J=l (2) where A is the regularization parameter, and lVv(i, j) I and IVU(Z, j)l represent the magnitude of the gradient of the ver- tical and horizontal components of the DVF, respectively. The addition of the (~) terms in Eq. (2) imposes a level of smoothness on the motion field that minimizes the DFD. In traditional (Tikhonov) regularized DVF estimation, both rj and ~ are chosen to be quadratic. As shown in Section 5., unfortunately (unweighed) Tikhonov regularized mo- tion estimators tend to over-smooth important discontinu- ous features of the DVF. For this reason, the management of discontinuities is an active area of research in motion estimation. In Konrad and Dubois [1] and Brailean and Katsaggelos [2] a ‘line process’ was used to adapt their motion models in the presence of abrupt motion boundaries in a Markov random field (MRF) formulation. Black and Anandan in [3] first introduced the use of robust measures in Eq. (2). They proposed robust non-convex measures for rj and # and demonstrated the relationship bet ween their robust approach and M RF for- mulations with a ‘line process’. Although significantly im- proved results are obtained in both formulations, the crite- rion to be minimized in the MRF formulation [1, 2] and in the robust formulation of [3] is non-convex, and have many local minima requiring costly minimization techniques that only approximate e a globally optimum solution. In [4] the authors considered robust non-convex ~ (@ quadratic) in an iterative half-quadratic regulariz at ion algorithm. The lim- ited performance of their approach was attributed to the non-convex selection of ~ which only allowed for an ap- proximation of the globally optimal DVF. In recent work, Rouchouze et. al. [5] and later Tull and Katsaggelos [6] suggested robust, convex@ functions which are capable of edge preserving regularization. Selecting @ and 4 to be convex allows one to obtain a globally opti- mal DVF estimate. The mechanism for edge preserving regularization in this framework can be seen by taking the derivative of Eq. (2) with respect to U(Z, j) which gives, 8Q(u, v, A) aprt(i,j)l Zirl(i, j) = #( DFD(i, j))fj +A ~ W(i,.i) ~u(z, ~, , e,j~e (3) C=l j=] ., where, @ is the support over which the derivative is non- Copyright (C) 1996 IEEE. All Rights Reserved.