IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 40, NO. 2, FEBRUARY 2002 405 Cloud Tracking by Scale Space Classification Dipti Prasad Mukherjee and Scott T. Acton, Senior Member, IEEE Abstract—The problem of cloud tracking within a sequence of geo-stationary satellite images has direct relevance to the analysis of cloud life cycles and to the detection of cloud motion vectors (CMVs). The proposed approach first identifies a homogeneous consistent cloud mass for tracking and then establishes motion cor- respondence within an image sequence. In contrast to the cross- correlation based approach as adopted in automatic CMV detec- tion analysis, a scale space classifier is designed to detect cloud mass in the source image taken at time t and the destination image at time . Boundaries of the extracted cloud segments are matched by computing a correspondence between high curvature points. This shape based method is capable of tracking in the cases of rotation, scaling, and shearing, while the correlation technique is limited to translational motion. The final tracking results pro- vide motion magnitude and direction for each contour point, al- lowing reliable estimation of meteorological events and wind veloc- ities aloft. With comparable computational expense, the scale space classification technique exceeds the performance of the traditional correlation-based approach in terms of reduced localization error and false matches. Index Terms—Cloud tracking, image classification, motion cor- respondence. I. INTRODUCTION S PATIO–TEMPORAL analysis of meteorological events is an important part of routine numerical weather analysis. In that context, a cloud tracking method is presented here for a sequence of geostationary satellite images. Given a pair of re- motely sensed images, captured at a fixed time interval (typ- ically, 30 min), the objective is to derive motion vectors as- sociated with the cloud mass. This correspondence process is a useful precursor to cloud motion vector (CMV) studies and spatio–temporal analysis of cloud life cycles. The spatio–tem- poral life cycle includes the generation, dissipation and assim- ilation of clouds that can be observed in a sequence of geosta- tionary satellite images. Cloud tracking is an example of motion analysis of de- formable shapes in a sequence of monocular images. In the absence of any meteorological disturbance, an assumption of path and shape coherence of the cloud mass between subse- quent image frames is tenable given a relatively short ( 1 h) duration of observation. The tracking of clouds involves two major subproblems. First, a homogeneous cloud mass needs to be detected for tracking. The detection of the cloud mass boundary is essen- tially a problem of image segmentation, and a scale-sensitive image classification approach is introduced in this paper to Manuscript received June 2, 1999; revised November 27, 2001. D. P. Mukherjee is with the Electronics and Communication Sciences Unit, Indian Statistical Institute, Calcutta, India 700035 (e-mail: dipti@isical.ac.in). S. T. Acton is with the Department of Electrical Engineering, University of Virginia, Charlottesville, VA 22904 USA (e-mail: acton@virginia.edu). Publisher Item Identifier S 0196-2892(02)01875-2. achieve this objective. The classification is performed on a scale space representation of the satellite imagery generated using area-based morphological operators. This classifier detects homogeneous cloud segments with minimum intra-segment classification error. The scale space approach gives improved segment integrity over the fixed-scale approach. The second subproblem is the evaluation of motion vectors. In this case, feature points are specified on the boundary of an ex- tracted cloud segment. These feature points are points of inflec- tion on the contour that remain almost stable (maintain curva- ture properties) over a brief period of observation. A cost func- tion that enforces both path and shape coherence properties of these feature points is minimized in order to establish correspon- dences between feature points. The organization of the paper is as follows. In the next section we review the related work in the literature. This is followed by a description of the image segmentation technique in Section III. Section IV presents the process of generating motion vectors. In Section V, results of cloud tracking from the geostationary satellite images are provided, followed by concluding remarks. II. BACKGROUND Cloud classification techniques have attracted considerable attention. A review of such schemes is provided in [18]. A multispectral classification of meteorological satellite images using a pixel-clustering algorithm is described in [19]. A number of cloud segmentation approaches have used pixel-wise neural-network-based classifiers that exploit textural and spectral features [13], [20], [25], [30]. In contrast, we have developed an area-based, shape-preserving morphological segmentation scheme for cloud extraction. Since our primary objective is cloud tracking, we are interested in detecting a homogeneous cloud mass which remains almost consistent during the period of observation and then extracting the cloud segment boundary. Using a pixelwise classification with a predetermined set of cloud classes results in a number of extraneous segments (minor regions) that lack significance for cloud tracking. Such details also increase the occurrence of intrasegment classification error. The proposed area mor- phological operators generate a scale space. The scale space contains a coarse-to-fine collection of image representations, for the input imagery, where the scale is defined as a function of the area of connected components within image level sets. The conventional classification schemes are adapted to classify the scale space of the remotely sensed imagery. In Section III, a detailed description of the segmentation process is given. Cloud tracking is of interest to meteorologists particularly in the estimation of wind velocity. Furthermore, tracking could provide valuable information with respect to growth, dissipa- tion and disintegration of cloud masses that transport pollu- 0196–2892/02$17.00 © 2002 IEEE