Robust Stereo Analysis zyxw K. Palaniappan, Yan Huangt, Xinhua Zhuangt, and A. Frederick Haslert Univesities Space Research Association tLab. for Atmospheres, Code 912 NASA/Goddard Space Flight Center Greenbelt, MD 20771 Abstract 1 Introduction One of the most difficult aspects of developing com- putational algorithms for stereopsis that match the intrinsic capabilities of human vision is the correspon- dence problem; that is locating the same point, if it exists, in multi-viewed time-varying sensor measure- ments. Correspondences have been determined us- ing feature-based or region-based matching algorithms with bottom-up or top-down implementations [3]. The bottom-up or low-level approach for stereo analysis in- cludes: i) extracting feature points or area measures in both views, ii) matching the feature points or area measures under certain geometric, illumination, re- flectance and object constraints, and iii) computing a depth or height map using the disparity values from correspondences using sensor geometry and scanning configuration. Most stereo algorithms invariably pro- duce errors due to noise, low image or feature content, geometric distortion, depth discontinuities, occlusion, illumination and reflectance changes across the scene and between views, transparency effects leading to multiple matches, and instability of the cameras and sensors during image formation. Such model viola- tions are difficult to handle in a comprehensive fash- ion. Robust statistical methods have recently been applied to a variety of computer vision problems in- cluding motion estimation [lo] [ll][S], surface recovery from range data zyxwvutsrqp [9], and image segmentation [2]. Ro- bust methods offer a powerful alternative to smooth- ness and regularization constraints to mitigate the ef- fects of model errors. A new multistage adaptive ro- bust (MAR) algorithm combined with a multiresolu- tion coarse-to-fine matching model is developed for ro- bust stereo analysis. Stereo analysis of remotely sensed images is use- ful for a variety of applications including cloud height measurement and digital terrain models. Stereopsis both in human vision and in remote sensing relies on the same principal of parallax but the matching problem is made more difficult due to the extremely long baselines with satellite geometries. The esti- mation of cloud-top structure using multiple satellite views (both geosynchronous and low earth orbiting) is an extremely challenging problem due to the time- dependent dynamics of the satellite-based imaging in- +Dept. of Electrical and Computer Eng. Univ. of Missouri Colum.iba, MO 65211 struments and complex fractal-like surface properties of clouds [7]. Robust estimation methods have the advantage of possessing adaptive properties to local depth changes in the prescence of outliers. Time se- quential stereoscopic observations of clouds from me- teorological satellites provide a basic analysis tool for a broad spectrum of applications [4] including numer- ical weather prediction, cloud modeling, and global climate understanding. A two-step robust stereo analysis algorithm is de- veloped and applied to the satellite-based cloud height estimation problem. The first step involves estimating initial disparities (depth map or height field provided rithm that uses regularization and image warping and is described in the next section. For the second step the matching model using to derive robust estimators is described in Section 3, followed by a description of the multi-stage robust estimation process for handling outliers and irregularities in the initial stereo disparity field. The incorporation of a multistage robust statis- tical process using a general matching model leads to improved performance. by a hierarchical coarse-to-fine stereo ana / ysis algo- 2 Parallel Automatic Stereo Analysis The first step involves estimating an initial set of dense disparities using a stereo pair of images. An Automatic Stereo Analysis (ASA) algorithm [5] [8] has been developed and implemented on the commer- cially available massively parallel supercomputer Mas- Par MP-2 at the NASA Goddard Space Flight Cen- ter. The parallel implementation enables the estima- tion and visualization of cloud surfaces interactively in realtime [7]. The ASA uses a hierarchical approach with a coarse-to-fine resolution implementation based on varying the size of the template windows (match- ing blocks) rather than filtering the image. The ASA can search for both horizontal and vertical dispari- ties though for the experiments in this paper the dis- parities have been constrained to be along horizontal epipolar lines only. Suppose that the stereo image pair consists of a reference image fr and a test image zyx ft. The ASA algorithm includes the following steps: Algorithm zyxw 175 0-8186-7190-4/95 $4.00 zyxwvutsrqpon 0 1995 IEEE