Applied Intelligence 18, 27–35, 2003 c  2003 Kluwer Academic Publishers. Manufactured in The Netherlands. A Novel Self-Organizing Neural Network for Motion Segmentation GIANSALVO CIRRINCIONE University of Picardie CREA, 33 rue Saint Leu, 80039 Amiens, France exin@dag.it MAURIZIO CIRRINCIONE CERISEP-CNR, viale delle Scienze snc, 90128 Palermo, Italy nimzo@cerisep.pa.cnr.it Abstract. Many computer vision techniques, above all for structure from motion problems, require a segmentation of the images captured by one or more cameras. This paper deals with the segmentation based on the motion information, but can be easily extended to other cases (color, texture and so on). A new neural network, the EXIN Segmentation Neural Network (EXIN SNN) is here introduced. It is incremental, self-organizing and considers its task as the solution of a pattern recognition problem. This original approach overcomes the limits of the traditional segmentation techniques, namely the need of a spatial support for the image objects and the translation parallel to the image plane for the objects in the scene. Examples are given both for synthetic and real images. Keywords: motion segmentation, neural networks, self organization, structure from motion 1. Introduction One of the most important topics in computer vision is the structure from motion (SFM) problem, i.e. the recovery of the motion and scene parameters from a sequence of images, captured by one or more sensors. The solution methods often require a segmentation of the image. Segmentation is also important for detec- tion tasks, e.g. for alarm systems. Motion-based seg- mentation refers to partitioning an image into regions of homogeneous 2D apparent motion. The apparent (projected) motion which is to be segmented not only depends on the 3D motion of the scene and on the cam- era motion, but also on the position of every point in the scene, and all these quantities are generally unknown. The motion-based segmentation can be expressed as the need for the determination of the motion within each region (possibly described by a motion model), the spatial support of each region and the number of regions. While segmentation is in itself a difficult is- sue, segmentation based on motion suffers from the fact that motion observations are partially hidden variables. The segmentation methods lie in (or between) two groups; those detecting flow discontinuities (local operations) and those detecting patches of self- consistent motion according to set criteria (global mea- surements). The first approach is very sensitive to noisy flow measurements. The methods of statistical regular- ization and image transformation fall somewhere in between these groups. Much of the segmentation re- search relies on the assumption that object motion is restricted to translational motion. The different tech- niques can be based on local measurements [1, 2], on analytic image transformations [3–5] and on reg- ularization [6–8], which are usually computationally expensive. They can also be based on simple cluster- ing or inconsistency with background flow [9, 10] and on globally organized clustering, where optical flow is used to segment the image into regions of uniform flow or uniform expansion, rotation or shear [11, 12]. Strangely, there are very few neural techniques even if the segmentation problem can be viewed as an unsu- pervised clustering problem in a feature space derived from the motion information and so is well suited for