A Multi-Target Tracking and GMM-Classifier for Intelligent Vehicles Cristiano Premebida and Urbano Nunes, Member, IEEE Abstract— Intelligent vehicles need reliable information about the environment in order to operate with total safety. In this paper we propose a flexible multi-module architecture for a Multi-Target Detection and Tracking System (MTDTS) complemented with a Bayesian object Classification layer based on finite Gaussian Mixture Models (GMM). The GMM pa- rameters are estimated by an Expectation Maximization (EM) algorithm, hence finite-component models were generated based on feature-vectors extracted from object’s classes during the training stage. Using the joint mixture Gaussian pdf modelled for each class, a Bayesian approach is used to distinct the object’s categories (persons, tree-trunks/posts, and cars) in a semi-structured outdoor environment based on data from a laser range finder (LRF). Experiments using real data scan confirm the robustness of the proposed architecture. This paper investigates a particular problem: detection, tracking and classification of objects in cybercars-like outdoor environments. I. INTRODUCTION I N the context of cybercars (www.cybercars.org), or in more general ITS and advanced driver assistance system (ADAS) technologies, applications on safe navigation, path following, platooning, obstacle avoidance, collision warning, object/target detection and classification, or a combination of the previous tasks, are typical and interesting problems to be solved. Several works have been done on using laser-scanners in multiple target tracking and classification. In order to classify objects, a voting scheme [4] and a multi-hypotheses approach [5] are examples of proposed methods. Vision based systems are also widely used for object/pedestrian detection [6],[9],[10]. In our case a multi-target detection and tracking system (MTDTS), complemented with a classification module, is proposed to handle the tasks of object detection, tracking and classification in outdoor semi-structured environments (cybercars-like scenarios). A flexible multi-module scheme to deal with this situation is presented, where each module is designed to perform a pre-determined task in a specific manner, but taking into account the whole system (multi- dependency framework). Although this architecture allows the use of other sensors, the results presented in this paper are only from using a laser range finder (LRF), mounted on a vehicle platform. This work was supported in part by Portuguese Science and Technol- ogy Foundation (FCT), under Grant POSC/EEA-SRI/58279/2004, and by CyberC3 project (European Asia IT&C Programme). C. Premebida is supported by the Programme AlBan, the European Union Programme of High Level Scholarships for Latin America, scholarship n o E04M029876BR. C. Premebida and U. Nunes are with University of Coimbra, Polo-II, and also with the Institute for System and Robotics, ISR, Coimbra, Portugal. {cpremebida;urbano}@isr.uc.pt Instead of discussing the details of each module under a general situation, this paper investigates a particular problem: automatic detection-tracking and classification of a set of ob- ject’s categories of interest (persons, trees/posts and cars) in outdoor environments using data from a 2D LRF. Hence, the major effort of this work has been focused on some modules that we think are more critical and relevant: tracking, data association and classification modules. In order to make the paper more tractable, a concise overview of our framework and the primary modules de- scription are presented in section 2. Section 3 addresses the tracking and data association, and section 4 is dedicated to object classification. Finally sections 5 and 6 present the experimental results and conclusions, respectively. II. MAIN ARCHITECTURE DESCRIPTION Our experimental platform is a bi-steerable four wheel au- tonomous vehicle. The control system is composed by three main subsystems [3], which are designated by path-following controller (PFC), vehicle’s pose estimator (VPE) and multi- target detection and tracking system (MTDTS). The latter subsystem works in an independent PC connected by a CAN bus throughout the main-system and constitutes the scope of this paper. The essential components of our proposed MTDTS architecture include the modules: data acquisition, segmentation (and pre-filtering), feature extraction, tracking and data association, object classification, a data context-base repository, and a coordinate updating feedback (see Fig. 1). A. Primary Modules Data Acquisition, Coordinate Updating, Pre- filtering/Segmentation, and Feature Extraction, henceforth called Primary Modules are depicted as white-blocks in Fig. 1, and will be described in this section. Data acquisition module is constituted by a 2D LRF, connected through a dedicated CAN bus using a Microcontroller-based RS422-to-CAN bus convertor module. The scan data transfer rate is approximately 36Hz. Fig. 1. MTDTS system architecture overview in Proc. of the IEEE Intelligent Transportation Systems Conference 2006 (ITSC 2006), Toronto, Canada, 2006.