VEHICLE THREAT LOCATING VIA THE DETECTION OF ANOMALIES ON ROADS AND THEIR VERGES Roel Heremans AGT R&D Group GmbH Hilpertstrasse 35 64295 Darmstadt, Germany email: RHeremans@agtinternational.com Wim Mees Royal Military Academy of Belgium Communication, Information, Systems and Sensors (CISS) Renaissancelaan 30 1000 Brussels, Belgium email: wim.mees@rma.ac.be ABSTRACT Situational awareness originating from advanced sensor systems augments the ability of sound decision making. This paper considers the image analysis problem given a multi-sensor system mounted on an army patrol vehicle that serves as an early threat alert mechanism. Hence, it concerns forward looking sensors that move through a complex environment whilst detecting events or points of interest. The objective comprises identifying potential threats to the vehicle such as Improvised Explosive De- vices (IED). This paper focuses on how an optical sensor and a forward-looking infrared (FLIR) are exploited for de- tecting and tracking stationary anomalies on the road’s sur- face and verges. The applied approach consists of a road scene analysis that extracts the road’s surface and verges followed by anomaly detection in each of the extracted re- gions separately. The detected anomalies from both sen- sors are tracked and combined in geographical coordinates where their threat-levels also increase due to the response of other sensors. The proposed methodology was assessed during a full system’s demonstration. The obtained results within a simplified real situation show considerable poten- tial. Scene segmentation, Road extraction, Anomaly detection 1 Introduction Situational awareness refers to the perception of envi- ronmental elements with respect to time and space, the comprehension of their meaning, and the projection of their status. It is concerned with perception of the envi- ronment critical to decision-makers in complex, dynamic areas. Situational awareness is an important facet of most application domains, e.g. surveillance, autonomous vehicles, advanced driver’s assistance systems and so forth. Consequently, research communities continuously endeavor augmenting situational awareness. The use of multi-sensor systems presents the opportunity of combin- ing the advantages of different sensors whilst minimizing their drawbacks. Within the framework of the Surveillance in an Urban Environments using Mobile Sensors (SUM) project, a system is constructed with as primary objective providing the operator with decision support information concerning potentially dangerous objects that lurk in and on the road’s surface and verges. Well-known threats to armed forces on patrol are the IED. The one characteristic that most of these IEDs share is their explosive nature. The literature reports on several sensing technologies for detecting explosive materials, e.g. ground penetrating radar, and infra-red sensors. The SUM framework adopts a dedicated sensor array that consists of four dissimilar forward looking sensors. The idea is raising detection accuracy whilst lowering the false alarms by extracting evidence of potential threats from each sensor individual and fusing the results. Furthermore, using diverse sensors quells the problematic detectability under different lighting conditions. The focus of this paper is on locating potentially dangerous objects using the output of the optical sensor and the FLIR separately. IEDs or other hazards do not really exhibit unique characteristics. Thus the detection of IEDs from optical data implies anomalies. These are of particular interest since they often correspond to significant information. Detecting outliers remains an important topic in numerous application domains. Chandola et al. [3] and Hodge et al. [8] present extensive surveys on this topic. For anomaly detection applied in the domain of forward looking sensors mounted on a moving platform, the literature is less rich. In [1], [5] and [12], a set of anomaly detectors aim at detecting explosive hazards using a FLIR. The detection of temporal changes is a related but dissimilar topic. It includes the detection of abandoned objects on road’s verges [10]. In the context of the SUM project, the anomaly detection follows an unsupervised approach. It consists of two consecutive steps. A first step segments the road scene in regions that are expected to exhibit similar characteristics. The second step identifies anomalous areas within these regions. Once anomalous areas are identified, their geographical location is approximated. The anomaly is considered as threat evidence and clusters of threat evidence are likely to correspond to physical objects. Proceedings of the IASTED International Conference Computer Graphics and Imaging (CGIM 2013) February 12 - 14, 2013 Innsbruck, Austria DOI: 10.2316/P.2013.797-030 KEY WORDS 54