Real-time detection of moving objects in a dynamic scene from moving robotic vehicles A. Talukder, S. Goldberg, L. Matthies, A. Ansar Jet Propulsion Laboratory, California Institute of Technology Pasadena, CA. Tel. (818)354-1000 – Fax (818)393-3302 Email: [ashit.talukder, steve.goldberg, larry.h.matthies, adnan.iansar]@.jpl.nasa.gov Abstract Dynamic scene perception is currently limited to de- tection of moving objects from a static platform. Very few inroads have been made into the problem of dynamic scene perception from moving robotic vehicles. We dis- cuss novel methods to segment moving objects in the mo- tion field formed by a moving camera/robotic platform in real time. Our solution does not require any egomotion knowledge, thereby making the solution applicable to a large class of mobile outdoor robot problems where no IMU information is available. We address two of the toughest problems in dynamic scene perception on the move, using only 2D monocular grayscale images, and secondly where 3D range information from stereo is also available. Our solution involves optical flow computa- tions, followed by optical flow field preprocessing to highlight moving object boundaries. In the case where range data from stereo is computed, a 3D optical flow field is estimated by combining range information with 2D optical flow estimates, followed by a similar 3D flow field preprocessing step. A segmentation of the flow field using fast flood filling then identifies every moving object in the scene with a unique label. This novel algorithm is expected to be the critical first step in robust recognition of moving vehicles and people from mobile outdoor ro- bots, and therefore offers a general solution to the field of dynamic scene perception. It is envisioned that our algorithm will benefit robot scene perception in urban environments for scientific, commercial and defense ap- plications. Results of our real-time algorithm on a mobile robot in a scene with a single moving vehicle are pre- sented. Keywords: Computer vision, dynamic scenes, moving object detection, optical flow, robotics, segmentation, scene understanding. 1. Introduction The robotics community to date has mostly focused on autonomous robot operation in static scenes [1], or highly constrained dynamic scenes [2]. Autonomous operation in urban scenes however realistically involves operation in the presence of moving objects, either to avoid hitting or being hit by moving vehicles/people, or detect, track, and approach moving people and vehicles in the scene. Moving object detection from fixed cameras using opti- cal flow algorithms that employ region-based correspon- dence measures [3], or feature-based matching [3] have been discussed extensively. However, 2D optical flow information by itself is insufficient to locate moving ob- jects on the move due to the effective motion of back- ground pixels, as shown in Figure 1b. It is unrealistic, and uneconomical to stop a robot frequently to enable it to localize moving objects. Therefore, we propose new techniques that will allow moving object detection on the move in real time. We classify the problem of moving object detection into four categories: 1. Detection of moving objects from a static camera 2. Detection of moving objects from a moving cam- era with known egomotion (from IMU, etc.), and knowledge of 3D/depth/range information (from stereo) 3. Detection of moving objects from moving camera without any knowledge of egomotion, and knowl- edge of 3D/depth/range information (from stereo) 4. Detection of moving objects from moving camera without any knowledge of egomotion, and no knowledge of 3D/depth/range information As mentioned previously, prior work on moving object detection has mostly concentrated on the first category [3]. The last two scenarios where no egomotion informa- tion is available are the toughest problems to solve, and have not been addressed much in previous research ef- forts. In [4], the expected image motion was computed using vehicle odometry. In [5], a quadratic motion model (a) Input observed image (b) 2D Optical flow vectors Figure 1: Typical estimated 2D optical flow image of a moving car observed from a moving robot platform