A MULTI-FRAME OPTICAL FLOW SPOT TRACKER Jizhou Li, Christopher Gilliam, and Thierry Blu Department of Electronic Engineering, The Chinese University of Hong Kong Email: {jzli, cgilliam, tblu}@ee.cuhk.edu.hk. ABSTRACT Accurate and robust spot tracking is a necessary tool for quantita- tive motion analysis in fluorescence microscopy images. Few track- ers however consider the underlying dynamics present in biological systems. For example, the collective motion of cells often exhibits both fast dynamics, i.e. Brownian motion, and slow dynamics, i.e. time-invariant stationary motion. In this paper, we propose a novel, multi-frame, tracker that exploits this stationary motion. More pre- cisely, we first estimate the stationary motion and then use it to guide the spot tracker. We obtain the stationary motion by adapting a re- cent optical flow algorithm that relates one image to another locally using an all-pass filter. We perform this operation over all the im- age frames simultaneously and estimate a single, stationary optical flow. We compare the proposed tracker with two existing techniques and show that our approach is more robust to high noise and vary- ing structure. In addition, we also show initial experiments on real microscopy images. Index Terms— Spot tracking, stationary motion, fluorescence microscopy, optical flow, all-pass filters 1. INTRODUCTION Recent developments in fluorescence microscopy, such as improved optics, electronic imaging and new fluorescent probes [1, 2], have enabled biologists to observe and investigate biological systems, such as intracellular processes, at an unprecedented spatiotemporal resolution [3]. A major challenge, therefore, is to understand not just the spatial organization of biological systems but their spatiotem- poral relationship [4]. A key technique used in the analysis of this relationship is spot tracking [3] - following the position of a spot over a series of time frames. However, robust and accurate tracking is difficult due to high noise levels in microscopy images [2] and fast dynamics, such as Brownian motion [5]. Numerous spot tracking methods have been proposed for differ- ent biological applications, for example see [6, 7, 8, 9] to list but a few. In general, these tracking methods all follow the same proce- dure: preprocess the image data, detect the spots in each frame and then link the detected spots over time to create trajectories [2]. A survey of spot detection in fluorescence microscopy was presented in [5], however, a recent evaluation of spot trackers, designed for microscopy imaging, found no one method outperformed the others in all situations [3]. The more general problem of object tracking has also been extensively studied in image processing. A partic- ularly popular method, used in applications such as crowd analy- sis [10, 11], is the Kanade-Lucas-Tomasi (KLT) tracker proposed in [12]. The KLT Tracker is based on Lucas and Kanade’s optical This work was supported by grants from the Research Grants Council (RGC) of Hong Kong (AoE/M-05/12). flow algorithm [13]. For a comprehensive review of the state-of-the- art see [14, 15]. In this paper, unlike existing approaches in the spot tracking lit- erature [3, 6, 7], we propose a multi-frame optical flow tracker that exploits global motion characteristics present in biological systems. More precisely, we assume there exists a motion pattern that under- pins the dynamics within the system being imaged. We define this underlying motion as the main part of the motion between any two consecutive images in the sequence that does not change, i.e. the stationary motion field. This type of stationary motion has been ob- served in many organisms, such as crowds of people [16], flocks of birds [17] and bacteria [18]. In particular, in the study of the col- lective motion behavior of cells, the structural movement often ex- hibits stable flow dynamics which do not significantly change over the period of several frames [19]. Hence, we propose to improve accuracy and robustness by using this stationary motion to guide the spot tracking. Similar to [12], we use an optical flow algorithm to estimate the stationary motion. Instead of enforcing temporal coherence on a frame-by-frame basis [20], we estimate a single optical flow for the whole image sequence using an adapted version of the algorithm proposed in [21]. This algorithm consists of relating local changes in one image to another image using all-pass filters and then extracting the optical flow from the filters. Our approach is to perform this operation over all the image frames simultaneously. We compare the proposed method with two existing tracking techniques for synthetic image sequences, which mimic the images obtained from confocal microscopy. We also demonstrate the applicability of the method for the estimation and tracking of multiple spot movements in real fluorescence confocal microscopy images. 2. OPTICAL FLOW ESTIMATION USING LOCAL ALL-PASS FILTERS Recently, Gilliam and Blu [21] presented a novel algorithm for op- tical flow estimation; termed the Local All-Pass (LAP) algorithm. Instead of using the optical flow equation [13, 22], the authors esti- mate the flow using local all-pass filters. In this paper, we want to adapt the LAP to estimate a stationary motion field. Accordingly, before proceeding, we outline the main aspects of the algorithm. 2.1. Idea 1 - Shifting is all-pass filtering The central concept of the LAP algorithm is that a constant optical flow is equivalent to filtering with an all-pass filter. To observe this equivalence, consider two images, I1 and I2, relate by a constant optical flow, u=[ux,uy ] T . Assuming brightness consistency [23], the two images can be related as I2(x, y)= I1(x - ux,y - uy ), where (x, y) is the pixel coordinates. In the Frequency domain, this