TARGET-TRACKING IN FLIR IMAGERY USING MEAN-SHIFT AND GLOBAL MOTION COMPENSATION Alper Yilmaz Khurram Shafique Niels Lobo Xin Li Teresa Olson Mubarak A. Shah Unv. of Central Florida, Computer Science Dept. Orlando FL-32816 Unv. of Central Florida, Mathematics Dept. Orlando FL-32816 Lockheed Martin Corp. Orlando FL-32819 ABSTRACT In this paper, we present a new approach for tracking tar- gets in forward-looking infrared (FLIR) imagery taken from an airborne, moving platform. Our tracking approach uses the target intensity and the Gabor response distributions and computes a likelihood measure between the candidate and the model distributions by evaluating the Mean Shift Vector. When the Mean Shift Vector based tracker fails to locate the target due to large ego motion, we compensate the ego motion using a multi-resolution scheme, which employs the Gabor responses of two consecutive frames, and assumes a pseudo-perspective motion model. We present the experi- ments performed on an AMCOM FLIR dataset of the pro- posed tracking algorithm. 1. INTRODUCTION Tracking moving or stationary targets in closing sequences of FLIR imagery is a challenging subject due to both the low contrast of the target with the background and the high ego (global) motion. In low resolution imagery the lack of the texture and the shape information of the targets make tracking even harder. Most methods for tracking targets in FLIR imagery, use a common assumption of compensated global motion. Even under compensated global motion, the tracking results of these methods are not convincing. To compensate global motion, Strehl and Aggarwal [1] have used a multi- resolution scheme based on the affine motion model. The affine model has its limitations and for FLIR imagery obtained from an airborne sensor, it is unable to capture the skew, pan and tilt of the planar scene. More- over, their target tracker uses the assumption that the targets are brighter than the background and it tracks targets based on the expected mean and central moments of the targets, which are not stable measures for FLIR targets. Shekarforoush and Chellappa [4] also use an ego motion stabilization approach. Once very hot or very cold targets are detected, the stabilization and tracking is based solely on the goodness of the detection and the number of targets, i.e. if the number of targets is not adequate, or there is sig- nificant background texture, the system may fail to locate the target, therefore stabilization fails to correctly stabilize the image. Braga-Neto and Goutsias [6] have presented a method based on morphological operators for target detection and tracking in FLIR imagery. Their tracker is based on the as- sumptions that the targets do not vary in size, they are either very hot or very cold spots, and there is small ego-motion. However, these assumptions contradict general closing se- quence FLIR imagery. Davies et al. [3] proposed a multiple target tracker sys- tem based on Kalman filters for FLIR imagery. The method assumes constant acceleration of the target, which is not valid for maneuvering targets. In addition, the method works only for sequences with no global motion. In this paper, we present a new approach for real-time tracking of the FLIR targets in presence of high global mo- tion. Compared to previous methods, the proposed approach does not use any constraints on the brightness of the target. Moreover, it is not required to have a target that has constant speed or acceleration. Our tracking algorithm is composed of two major modules. The first module is based on min- imizing the distance between the statistical distributions of the target and the target candidate. The statistical distribu- tions are obtained from Gabor filter responses and the inten- sity of the frames. We used Gabor filter response of the im- ages since these 2-D quadrature phasor filters are conjointly optimal in providing the maximum possible resolution for information about the orientation and spatial frequency con- tent of local image structure [8]. If the tracker module fails to locate the new target position due to high ego motion, the second module compensates the global motion. The global motion estimation module uses a multi-resolution scheme of [7] assuming a planar scene and using perspective pro- jection for image formation. It uses Gabor filter responses of two consecutive frames to obtain the pseudo perspective parameters. The global motion compensation module is ex- ecuted when the mean-shift tracker fails to track the target.