Autonomous perception techniques for urban and industrial fire scenarios. J. Capitán, D. Mantecón, P. Soriano and A. Ollero Robotics, Computer Vision and Intelligent Control Group Escuela Superior de Ingenieros, Camino de los Descubrimientos s/n, 41092 Sevilla, Spain [jescap, mantecon, psoriano, aollero]@cartuja.us.es Abstract—This paper presents autonomous perception techniques for disaster management and particularly for urban and industrial fire scenarios. The perception from static and mobile (on-board UAV) cameras is considered for smoke detection and cooperative tracking The algorithms used to achieve these functionalities will be described. The implementation of these techniques in the AWARE platform and the application in the general AWARE experiments to detect smoke and track firemen of the Seville fire brigades will also be presented. Keywords: Computer vision, Cooperative Perception, fire scenarios, smoke detection, person tracking. I. INTRODUCTION This paper deals with the development of autonomous perception functionalities to be used in disaster management scenarios. Particularly, the paper considers two different valuable functionalities in urban and industrial fires: smoke detection and tracking of persons. Disaster management scenarios usually deal with dynamic environments and varying conditions for perception. The robustness and reliability of autonomous perception in these scenarios are main issues. In many cases a single autonomous entity (i.e. robot or camera) is not able to acquire all the information required for the application because of the characteristic of the particular task or the harmful conditions (i.e. loss of visibility). Then, in these scenarios, the cooperation of several of these entities is relevant. The work described in this paper has been carried out in the European Commission project AWARE on the Autonomous self-deploying and operation of Wireless sensor-actuator networks cooperating with Unmanned Aerial vehicles (UAVs). (http://www.aware-project.net). Although the project consists of the development of the whole platform and involves issues related to the cooperation and communication among the different UAVs and Ground Sensor Networks (GSNs), this paper focuses on the Perception System (PS). This system is based on building and updating a consistent representation of the environment suitable to achieve detection and tracking using the sensors provided by the AWARE platform: visual and infrared cameras, sensors of scalar magnitudes such as temperature, humidity, CO or radio signal strength. The AWARE PS system has been designed to detect events such as fires and to perform tracking of firemen and vehicles in the fire scenario by using the information from Ground Camera Nodes (GCN), cameras mounted on autonomous helicopters, and nodes of the wireless sensor network (the firemen also carry nodes of this network). Figure 1 illustrates the tracking. A distributed architecture has been adopted for the PS in AWARE. This architecture decreases the requirements on data transmission and improves the scalability of the whole system. In addition, it will be possible to divide the processing load among different perception nodes. These nodes will process locally the environment information (images, sensor readings, ...) in order to decrease the amount of data transferred through the network. All the nodes will share the perception information extracted from their local sensors and then, improve its knowledge of the environment by integrating the measurements from other nodes. AWARE will be demonstrated on fire detection and monitoring in urban scenarios, and on tracking the position of people and vehicles. This paper presents results on autonomous smoke detection, which is a basic functionality in urban fires. Furthermore, the paper presents results on the cooperative tracking of firemen. These results have been obtained in the first general experiments of AWARE carried out in March 2007. The paper is organized as follows; Section II describes perception techniques developed to achieve the smoke detection functionality. Section III and IV detail the firemen detection and cooperative tracking algorithms respectively. Section V is devoted to present some experiments. Finally, section VI gives some conclusions and future work. Fig. 1. Tracking of persons using ground cameras, UAV and nodes of a wireless sensor network. Proceedings of the 2007 IEEE International Workshop on Safety, Security and Rescue Robotics Rome, Italy, September 2007 978-1-4244-1569-4/07/$25.00 2007 IEEE.