IMTC 2003 - Instrumentation and Measurement Technology Conference Vail, CO, USA, 20-22 May, 2003 Camera Based Motion Tracking for Data Fusion in a Landmine Detection System Wannes van der Mark, Johan C. van den Heuvel, Eric den Breejen TNO Physics and Electronic Laboratoy Electro-Optical Systems, Oude Waalsdorperweg 63, The Hague, The Netherlands Phone: +31-70-374-0375, Email: vandermark,vandenheuvel,breejen @fel.tno.nl Frans C.A. Groen University of Amsterdam Informatics Institute, Kruislaan 403, Amsterdam, The Netherlands Phone: +31-20-525-7461, Email: groen@science.uva.nl Abstract – We present a method, based on stereo vision, for estimating the position and orientation of the LOTUS platform and its sensors. The LOTUS platform was developed to demonstrate the capabilities of automated landmine detection. Motion of the platform and pose and position of its sensors have to be measured to relate different sensors observations accurately to each other. Techniques from camera calibration are used for sensor pose and position estimation. The platform motion is estimated from tracked features on the ground. A special estimator was developed to deal with problems related to estimating rotation from coplanar surface points. This estimator also uses weights in order to remove outlier points caused by tracking errors and other influences. Simulation experiments shows that the weights can safeguard the estimator against a limited amount of outliers. Experiments with real stereo images from the LOTUS platform show that the relative pose and position of sensor can be estimated with high accuracy. When combined with the ego-motion of the cameras the position of the sensors can be related to fixed points to the ground. The results show that the vision based approach provides more useful position estimates when compared to an odometry based approach. I. INTRODUCTION For safety reasons, there are dangerous tasks which can be better performed by machines then by men. An example of such a task is clearance of landmines in former conflict areas. The main problem of demining has always been finding the buried mines. Currently this is done mainly by hand by the demining personal. Because of the sheer number and areas in which the landmines still remain, this is a very tedious and dangerous task. The LOTUS project has been a cooperation between several European partners to develop technology for automated land- mine detection. Result of their collaboration has been the LO- TUS platform, which was used to demonstrate the capabilities of automatic landmine detection and marking for humanitarian demining operations in Bosnia. On the platform multiple sensors are present which are able to detect properties which may belong to landmines. This ap- proach was chosen because a practical system must be able to find a large variety of landmine types. As can be seen in Fig. 1. the Lotus consists of a all-terrain vehicle with a large metal frame on which all the sensors have been mounted. Three types of sensors are used to detect landmines; ground pene- trating radar, metal detector and cameras. The ground penetrating radar (GPR) has been mounted near the vehicle. This radar was developed by EMRAD (United Kingdom). It uses 16 antennas to measure the density of the ground for different depth layers. The metal detector (MD) is mounted at the front of the frame. This sensor which was pro- duced by Froestner (Germany) contains 7 detector coils. The coils measure if metal containing parts are near. High in the frame, between the MD and the GPR, downward looking in- frared and multi spectral (IR) cameras are mounted. These cameras observe temperature differences in ground tempera- ture. In order to indicate detected landmines on the ground a marking unit is used at the back of the vehicle. This is a device which can mark a part of the terrain with spray paint. Data from the different sensors needs to be collected for the same location on the ground in order to decide if it contains a mine. The LOTUS drives over a minefield in a straight line. In this way, the MD will first pass over a location, then the IR cameras and finally the GPR. A virtual grid aligned with the ground plane is used to com- bine the different sensor measurements. The size of each grid cell is 25 mm square. In [1] several techniques for landmine detection with sensor fusion are described which are based on the grid approach. Currently, odometry is used to measure the progress of the platform over the lane. Together with the fixed distances between sensors this is used to incorporate observa- tions into the stationary grid. Field tests have shown that the current approach is suffi- cient on flat terrain. However, in more difficult terrain the motion platform is more complicated then a single translation. Also changes in orientation have to be considered because of the uneven surface. Localization is further complicated by the way the MD is mounted on the platform. To ensure good sen- sor readings, the sensor can follow the contours of the surface. However, in unstructured terrain the MD will be displaced by 0-7803-7705-2/03/$17.00©2003IEEE 817