Multi-Objects Segmentation and Tracking Based Graph-Cuts Amira Soudani Ezzeddine Zagrouba RIADI Laboratory, Team of research SIIVA. RIADI Laboratory, Team of research SIIVA University of la Manouba, Higher Institute of Computer University of la Manouba, Higher Institute of Computer 2 Rue Abou Rayhane Bayrouni 2 Rue Abou Rayhane Bayrouni 2080 Ariana, Tunisia 2080 Ariana, Tunisia amira.soudani@gmail.com ezzeddine.zagrouba@fsm.rnu.tn ABSTRACT In this paper we present an algorithm that joints segmenta- tion and tracking of multiple objects in a video via graph- cuts optimization technique. The proposed approach is com- posed of two steps. First, we initialize tracked objects through an initialization step based on background subtraction algo- rithm. Hence we obtain initial observations that will be used to predict the location of target object in the next frame. Then, we process a tracking step based on an energy function associated to each predicted observation. The minimization of this energy via graph-cut allows us to yield a better seg- mentation that matches extracted observations with initial detected objects. Experimental validation of the proposed method is performed in several video sequences and provides us significant tracking results. Keywords Tracking, Segmentaion, Graph-Cuts, Prediciton. 1. INTRODUCTION Object segmentation and tracking in video has been the fo- cus of many researches. In fact, robust and accurate sepa- ration of background from foreground objects has been con- sidered crucial in many applications. Several object tracking approaches have been proposed in this aim. In addition to the algorithm itself, the difference between those methods lies on the choice of the representation and shape of the tracked objects, on the property of the the image and on the nature of the estimated motion. This choice depends on the application and the processed video. Tracking algorithm can be classified on three categories. The first is the the Point Tracking methods that aim to match detected objects between successive images. There are de- terministic methods which associate observations to tracked objects by minimizing a distance computed based on some characteristics of the object (proximity and appearance) [1], [2], [3], [4]. We find also the probabilistic methods that deal with variations (noise, movement, appearance) by adding an incertitude to the models associated to the object and the observations [5].There are also methods based on the min- imizing of energy functions in order to follow a contour or a region taking into account the topology changes [6], [7], [8], [9], [10]. Secondly, there are Kernel Tracking methods which are based on the tracking of a predefined shape (rect- angle or ellipse) around or inside the tracked objects. They are based on the conservation of the appearance (usually color or luminance) of the object for at least two consecu- tive frames. Those methods can be based on the differential tracking of object which assumes a conservation of the lu- minance of visible pixels between two consecutive frames [11], [12], [13] or based on the tracking of color distribu- tions [14], [15]. Finally, there are the Silhouette Tracking methods which apply dynamic segmentation without prior knowledge about the object’s shape. They are based on suc- cessive segmentations and they generally evolve the edge of the object at the previous frame until its new position at the current one. Current approaches can be classified into methods based on state models that define a model for the object’s edge which will be considered as a state model for the filtering algorithm [16], [17], [18]. Each class of method cited above presents positive as nega- tive points. The Point Tracking methods can treat the case of apparition of new objects in the scene whereas the quality of track is depending well on the external detected observa- tions. Thereby, if they are not well detected the tracking process can fail. The Kernel Tracking methods allow ro- bust tracking with low cost but can’t treat the entrance of new objects on the scene and they are not adapted for the tracking of small objects. Finally, the Silhouette Tracking approaches whose main advantages are their flexibility to handle a large variety of object shapes and their capability for dealing with object split and merge. However they not deal with apparition of new objects on the scene. On the other hand, the graph-cuts optimization techniques has reached important results especially in image segmen- tation[19], [20]. Therefore, many works tried to adapt this technique on multi-objects tracking and it gives considerable results [21], [22], [23]. In this paper, we address the problem of multiple object segmentation and tracking. We will present an algorithm of