Improvements in Accuracy of Single Camera Terrain Classification Syed Muhammad Abbas, Abubakr Muhammad Department of Electrical Engineering School of Science and Engineering, LUMS Lahore, Pakistan {13060026,abubakr}@lums.edu.pk Syed Atif Mehdi, Karsten Berns Department of Computer Science University of Kaiserslautern Germany {mehdi,berns}@cs.uni-kl.de Abstract— Autonomous terrain classification is an important requirement for robotic applications for the outdoor and more so for off-road systems. Different technique have been developed in recent years mainly relying on either color features or on texture-based features for classification. We present an approach which combines the two approaches and delivers an overall increase in performance and accuracy. We describe the computational framework, training dataset, off-line learning and real-time classification results of our system. We report overall average classification accuracies in excess of 98% in a fair experimental setup along with confusion matrices. Our method gives a noticeable improvement in accuracy for classify- ing similar terrain classes over the current state of the art that uses only texture for classification with acceptable overhead for real-time applications. I. I NTRODUCTION As outdoor robotics is evolving, more and more chal- lenging tasks are being assigned to robots [1]. One of the most challenging task in an outdoor robotics application is to classify the terrain on which the robot is moving. An autonomous outdoor ground robot needs to know each and every feature around it in order to execute a complex task. By using only on-board sensors, we cannot identify environment fully. For indoor environments, normally we have smooth planar floors and walls. But in an outdoor environment, we can have different types of terrains built both by nature and humans. Intelligent selection of a traversable terrain during the execution of a particular task, guarantees success. Selecting a terrain can be helpful for a robot in avoiding a fail-state or even total loss. Sometimes the nature of the task itself demands the classification of terrain e.g. autonomous driving, autonomous land mines sweeping and planetary explorations etc. In these applications, the classification plays a direct role in successful completion. Terrain classification is done using different methods such as by using color feature information, texture features of the terrain and using depth information. In this paper we aim to do terrain classification using a single camera, in line with our overall goal of developing a very cost effective outdoor robot for off-road applications [3]. In this paper, we have proposed an approach for terrain classification which uses the best of both worlds i.e. both color features and texture features to decide the winning *This work has been supported by the German Academic Exchange Service (DAAD) with funds from the German Federal Foreign Office for the project titled ALVeDa. class. Some images are very rich in colors and it is wise to classify them with respect to color features as shown in Figure 1. Similarly, some images are high in texture so they should be classified with texture based features. Moreover, same terrain with different lighting conditions and viewing angle of the camera can make it vary in color features or texture features. So instead of dealing with all the images with the same approach, we have developed a system that suitably combines the classification of both texture and color based approaches. Experiments have shown an overall increase in the accuracy by combining the two classification approaches. Fig. 1. Left image is rich in color, Right image is high in features. In Section II, we have presented the literature survey for this task. In Section III, we give details of our approach. Section IV explains the offline experiments along with very detailed result analysis. Section V shows the real time application of our algoritm along with results. Lastly Section VI summerizes our conclusions. II. RELATED WORK Terrain classification started in the early 70s for satellite imagery mainly for military purposes. Later it has been used in civilian applications such as estimating total cultivated land, water resources, flood damage etc. using satellite imagery. As robotics evolved from simple predefined tasks to complex uncertian operations, many researchers have investigated the problem of terrain classification, mainly for wheeled autonomous robots. This domain gained a huge momentum after Mars Exploration mission and DARPA Grand Challenge in 2004 [2]. LIDARS and depth cameras are generally used to detect the obstacles in an outdoor / indoor environment for autonomous navigation. But classifying the cover of the terrain on which the robot is traversing is not possible with depth information only.