DETECTION, CATEGORIZATION AND RECOGNITION OF ROAD SIGNS FOR AUTONOMOUS NAVIGATION Aly A. Farag and Alaa E. Abdel-Hakim Computer Vision and Image Processing Laboratory University of Louisville, Louisville, KY 40292 E-mail: {farag, alaa }@cvip.uofl.edu http://www.cvip.uofl.edu Abstract In this paper we present a novel and robust approach for detection, categorization and recognition of road signs. It is known that the standard road signs contain few and easily distinguishable colors, such as red for prohibition, yellow for warnings, green, blue and white. We use a Bayesian approach for detecting road signs in the captured images based on their color information. At the same time, the results of the Bayes classifier categorize the detected road sign according to its color content. The SIFT transform is employed in order to extract a set of invariant features for the detected road sign label(s). Recognition is done by matching the extracted features with previously stored features of standard signs. We illustrate the accuracy and robustness of this approach. 1. INTRODUCTION The main objective of the project, we are working in, is to develop a smart autonomous vehicle that can navigate through an environment and, through a sensor suite, collect data about the environment which feeds into an on board intelligent system for understanding the environment and performing certain tasks of interest. One of the goals of our system is to provide a Driver Support System (DSS) that will be employed in a pedestrians aiding system. In such applications, road sign detection and recognition (RSR) is very important, since the road signs carry much information necessary for successful, safe and easy driving and navigation. The RSR approach, proposed in this paper uses a Bayes classifier to detect the road signs in the captured image (e.g., [1]), based on its color content. The color category of the road sign is very important in the recognition of it. For example, two identical signs with different colors may have a completely different interpretation because of the difference between their colors. The Bayes classifier does not just label the captured image only, but it categorizes the labels to the appropriate category of the road sings as well. Based on the results obtained by the Bayes classifier, an invariant feature transform, namely the Scale Invariant Feature Transform (SIFT) is used to match the detected labels with the correspondent road sign [2]. The contribution in this paper is in using an invariant feature approach (SIFT) for the RSR problem. Using the SIFT transform for the matching process achieves several advantages over the previous work in RSR. For example, it overcomes some difficulties with previous algorithms such as the slowness of template matching based techniques [3], the need for a large number of various real images of signs for training like the neural-based approaches [4], or the need for need a priori knowledge of the physical characteristics of the lighting illumination of the signs like in [5]. As another advantage of using the Bayes classifier is the acceleration of features extraction and matching operations of the SIFT transform by shrinking the matching area to the labels only. Also, it limits the search subspace of the SIFT transform by determining the color category of the detected sign. 2. RELATED WORK There are many researches in the literature deal with RSR problem. In this section, we will explore some of those approaches and show their advantages and their weak points, which are overcome by using our proposed approach. In [3], the authors used template matching for recognition of the road signs in the regions of interest (ROI) in the captured image. The ROI of the road image is determined by expecting the possible location(s) of the sign or by using the color information of the road image. The approach of [3] inherits the difficulties of the template matching schemes, namely, the relatively slowness and the need for various shapes for each template to consider different deformations resulted from changes in scale, orientation, rotation ...etc. In [6], the authors used Laplace kernel classifier for road sign classifications. They used the Laplace kernel classifiers in the decision tree. The smoothing parameters of the Laplace kernel are optimized by the This work is supported by US Army under grant DABT60-02-P-0063. Proceedings of Acivs 2004 (Advanced Concepts for Intelligent Vision Systems), Brussels, Belgium, Aug. 31-Sept. 3, 2004 125