Computer Vision and Image Understanding Vol. 73, No. 1, January, pp. 43–63, 1999 Article ID cviu.1998.0694, available online at http://www.idealibrary.com on Simultaneous Surface Approximation and Segmentation of Complex Objects Chia-Wei Liao and G´ erard Medioni Institute for Robotics and Intelligent Systems, University of Southern California, Los Angeles, California 90089-0273 E-mail: cliao@iris.usc.edu Received January 22,1997; accepted February 12,1998 Deformable models represent a useful approach to approximate objects from collected data points. We propose to augment the basic approaches designed to handle mostly compact objects orobjects of known topology. Our approach can fit simultaneously more than one curve or surface to approximate multiple topologically complex objects by using (1) the residual data points, (2) the badly fitting parts of the approximating surface, and (3) appropriate Boolean operations. In 2-D, B-snakes [3] are used to approximate each object (pattern). In 3-D, an analytical surface representation, based on the elements detected, is presented. The global representation of a 3-D object, in terms of elements and their connection, takes the form of B-spline and B´ ezier surfaces. A B´ ezier surface is used to connect different elements, and the connecting surface itself conforms to the data points nearby through energy minimization. This way, a G 1 conti- nuity surface is achieved forthe underlying 3-D object. We present experiments on synthetic and real data in 2-D and 3-D. In these experiments, multiple complex patterns and objects with through holes are segmented. The system proceeds automat- ically without human interaction or any prior knowledge of the topology of the underlying object. c  1999 Academic Press Key Words: snakes; deformable model; B-spline; segmentation; energy minimization. 1. INTRODUCTION The inference of an analytical surface from a cloud of points (boundary points of the object) is an important research area, because surface features can be made explicit under this repre- sentation. A deformable model, which fits surfaces to data points through minimization, is a good candidate for this purpose. The idea of fitting data by a deformable model in 2-D, known as a “snake,” can be found in the work of Kass et al. [1]. Such models are generalized to 3-D by the same authors [2] for sur- faces of revolution. Features, such as edges and curves, can be detected by optimizing models of applied forces (resulting from, for example, curve contrast) and smoothness. Snakes have been used in image analysis applications, which require edge, curve, and boundary segmentation. They are able to conform to object shapes, such as biomedical structures, from noisy observations. Due to their dynamical nature, snakes are useful to support the in- teraction between users and computers. A user can interactively edit the shape of an object by adjusting appropriate parameters. Working from a global viewpoint, snakes are quite robust to the presence of noise. Several variants of snakes exist which are based on Fourier descriptors [38, 39], B-splines [3, 40, 41], finite elements [42], balloons [43], and discrete representation [44, 45], and the lit- erature on 3-D deformable models is also rich [11, 19–37, 48]. Good results have been obtained, but these algorithms handle mostly topologically simple objects and might fail under the following two conditions: —First, there may be more than one underlying object, and these objects might be close to one another. It takes sophisticated segmentation to separate these mixed objects if the data are corrupted by noise. In this case segmentation is not a trivial job at all. —Second, they cannot handle objects with deep, narrow, or through cavities, especially when these are winding inside the objects. The reason that most deformable algorithms fail to cap- ture these cavities and holes is that they lack a good external energy definition to estimate the difference between the fitting surface and the underlying object, and a good (or right) initial guess, which is more important, of the object. The fitting process is bound to fail if the topology of the initial surface is wrong. For example, a genus 0 surface cannot approximate a torus ac- curately. How to infer the right topology for the initial surface might lead to a circular problem. Taubin et al. [15–17] used implicit algebraic curves and sur- faces, which assume the form of polynomials, to fit the collected data. They performed segmentation through factorization. The main difficulties are that the Euclidean distance between the data point and the implicit curve (or surface) is not easy to obtain when fitting. Szeliski et al. [18] presented a model based on dynamic par- ticles. By (1) adding intermolecular, coplanarity, conormality, and cocircularity potentials to the internal energy, (2) creating particles appropriately, and (3) doing triangulation, their model can handle arbitrary topology. The results are very encouraging. 43 1077-3142/99 $30.00 Copyright c  1999 by Academic Press All rights of reproduction in any form reserved.