170 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 50, NO. 1, JANUARY 2001 An Algorithm for Detecting Roads and Obstacles in Radar Images Kesav Kaliyaperumal, Sridhar Lakshmanan, and Karl Kluge, Member, IEEE Abstract—This paper describes an algorithm for detecting roads and obstacles in radar data taken from a millimeter-wave imaging platform mounted on a stationary automobile. Such an algorithm is useful in a system that provides all-weather driving assistance. Road boundaries are detected first. The prior shape of the road boundaries is modeled as a deformable template that describes the road edges in terms of its curvature, orientation, and offset. This template is matched to the underlying gradient field of the radar data using a new criterion. The Metropolis algorithm is used to deform the template so that it “best” matches the underlying gra- dient field. Obstacles are detected next. The radar returns from image pixels that are identified as being part of the road are pro- cessed again, and their power levels are compared to a threshold. Pixels belonging to the road that return a significant (greater than a fixed threshold) amount of incident radar power are identified as potential obstacles. The performance of the algorithm on a large all-weather data set is documented. The road edges and obstacles detected are consistently close to ground truth over the entire data set. A new method for computing the gradient field of radar data is also reported, along with an exposition of the millimeter-wave radar imaging process from a signal-processing perspective. Index Terms—All-weather vision, Bayesian detection, collision avoidance, radar backscatter, 77-GHz radar. I. INTRODUCTION T HIS paper addresses the problem of detecting roads and obstacles from radar data images obtained from an auto- mobile mounted imaging platform. Such radars, operating at millimeter wavelengths, have the ability to penetrate through rain, fog, snow, darkness, etc., and provide an “alternate” image of the scene in front of a vehicle. Fig. 1 shows two road scenes: one obtained in clear weather and the other under foggy con- ditions. Note that even though the road edges and the cars are difficult to discern in the foggy visual image, they are quite dis- tinct in the corresponding radar image. The radar image, how- ever, is difficult to interpret because its modality, resolution, and perspective are very different from visual images. Without ex- pert training, the position of the roads and obstacles in the radar image cannot be quickly determined. Therefore, the ability to automatically identify and extract roads and obstacles from the radar data and project them onto the normal human plane of Manuscript received May 20, 1999; revised February 25, 2000. This work was supported by the Department of Defense under DoD-DAAH04-95-1-0449 and DoD-DAAE07-96-C-150 and by the National Science Foundation under NSF-CDA9413862 and NSF-EEC9531589. K. Kaliyaperumal and S. Lakshmanan are with the Department of Electrical and Computer Engineering, University of Michigan–Dearborn, Dearborn, MI 48128-1491 USA (e-mail: kesavraj@umich.edu; lakshman@umich.edu). K. Kluge is with the Artificial Intelligence Laboratory, Univer- sity of Michigan, Ann Arbor, MI 48109-5439 USA (e-mail: kck- luge@eecs.umich.edu). Publisher Item Identifier S 0018-9545(01)04230-X. view is an enabling technology for all-weather driving assis- tance systems. Automotive radar imaging is a very active area of research and development within the intelligent transportation systems community and has been the central theme in a number of recent publications [6], [7], [9], [11], [14]–[16], [21]–[23]. In order to appreciate why the problem of extracting roads and obstacles from radar images is a difficult one, a close ex- amination of such images is necessary. Images obtained from millimeter-wave radars inherently have poor contrast. This is a direct consequence of the fact that the power level of the radar returns from much of the road scene is relatively small, except for a few isolated points. 1 1) The road itself forward scatters much of the incident radar signal and hence returns very little power back to the radar. 2) The sides of the road, because they are of a coarser struc- ture than the road, return a slightly higher amount of power, and 3) Man-made obstacles, such as automobiles, sign posts, fences, etc., which contain sharp polyhedral corners. re- turn a large amount of the incident power. Fig. 2 shows the radar image of a typical road scene. The lack of contrast difference between the road and the sides in the image is especially evident in comparison to the accompanying visual image of the same scene. This already meager contrast further degrades as the range from the radar increases. So, dis- cerning which portion of the radar image corresponds to the road is difficult. Although the radar image consists of large patches of homogeneous radar returns, there are many isolated points where the returns are very high. Several of these points are off the road and hence possess no potential danger to the driver. Therefore, what points constitute a legitimate obstacle to the ve- hicle is difficult to decide because of the associated risk of high false alarm. This paper presents an algorithm that finds roads and obsta- cles in millimeter-wave radar images of typical road scenes. It consists of four components. 1) Deformable Template: The shape of a typical road is mod- eled using a parabolic template, which parameterizes the road edges in terms of its curvature, slant, and offset. Var- ious instances of the roads shape are obtained by varying these parameters. 2) Matching Function: A new likelihood function is formu- lated to favor deformations of the template that place them 1 We refer the reader to [12], [25], and [26] for studies of the electromag- netic phenomenology that underlie the radar imaging process and for a precise mathematical explanation as to why various material surfaces reflect incident millimeter-wave radiation differently. 0018–9545/01$10.00 © 2001 IEEE