Survey of Pedestrian Detection for Advanced Driver Assistance Systems David Gero ´nimo, Antonio M. Lo ´pez, Angel D. Sappa, Member, IEEE, and Thorsten Graf Abstract—Advanced driver assistance systems (ADASs), and particularly pedestrian protection systems (PPSs), have become an active research area aimed at improving traffic safety. The major challenge of PPSs is the development of reliable on-board pedestrian detection systems. Due to the varying appearance of pedestrians (e.g., different clothes, changing size, aspect ratio, and dynamic shape) and the unstructured environment, it is very difficult to cope with the demanded robustness of this kind of system. Two problems arising in this research area are the lack of public benchmarks and the difficulty in reproducing many of the proposed methods, which makes it difficult to compare the approaches. As a result, surveying the literature by enumerating the proposals one-after-another is not the most useful way to provide a comparative point of view. Accordingly, we present a more convenient strategy to survey the different approaches. We divide the problem of detecting pedestrians from images into different processing steps, each with attached responsibilities. Then, the different proposed methods are analyzed and classified with respect to each processing stage, favoring a comparative viewpoint. Finally, discussion of the important topics is presented, putting special emphasis on the future needs and challenges. Index Terms—ADAS, pedestrian detection, on-board vision, survey. Ç 1 INTRODUCTION D UE to the rise in the popularity of automobiles over the last century, road accidents have become an important cause of fatalities. About 10 million people become traffic casualties around the world each year, and two to three million of these people are seriously injured [1], [2]. For instance, in 2003, the United Nations reported almost 150,000 injured and 7,000 killed in vehicle-to-pedestrian accidents just in the European Union alone [3]. Both the scientific community and the automobile industry have contributed to the development of different types of protection systems in order to improve traffic safety. Initially, improvements consisted of simple mechan- isms like seat belts, but then more complex devices, such as antilock bracking systems, electronic stabilization programs, and airbags, were developed. Over the last decade, research has moved toward more intelligent on-board systems that aim to anticipate accidents in order to avoid them or to mitigate their severity. These systems are referred to as advanced driver assistance systems (ADASs) [2], [4], [5], as they assist the driver in marking decisions, provide signals in possibly dangerous driving situations, and execute counteractive measures. Some examples are the adaptive cruise control, which maintains a safe gap between vehicles and the lane departure warning that acts when the car is driven out of a lane inadvertently. In this paper, we focus on a particular type of ADAS, pedestrian protection systems (PPSs). The objective of a PPS is to detect the presence of both stationary and moving people in a specific area of interest around the moving host vehicle in order to warn the driver, perform braking actions, and deploy external airbags if a collision is unavoidable (evasive actions could be an option if the pedestrian surroundings are sensed). Accident statistics indicate that 70 percent of the people involved in car-to-pedestrian accidents were in front of the vehicle, of which 90 percent were moving [6]. Therefore, PPSs typically use forward-facing sensors. The main challenges of a PPS involve detection of pedestrians. These challenges are summarized by the following points: . The appearance of pedestrians exhibits very high variability since they can change pose, wear different clothes, carry different objects, and have a consider- able range of sizes (especially in terms of height). . Pedestrians must be identified in outdoor urban scenarios, i.e., they must be detected in the context of a cluttered background (urban areas are more complex than highways) under a wide range of illumination and weather conditions that vary the quality of the sensed information (e.g., shadows and poor contrast in the visible spectrum). In addition, pedestrians can be partially occluded by common urban elements, such as parked vehicles or street furniture. . Pedestrians must be identified in highly dynamic scenes since both the pedestrian and camera are in motion, which complicates tracking and movement analysis. Furthermore, pedestrians appear at differ- ent viewing angles (e.g., lateral and front/rear IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 32, NO. 7, JULY 2010 1239 . D. Gero´nimo, A.M. Lo´pez, and A.D. Sappa are with the Computer Vision Center and the Computer Science Department, Universitat Auto`noma de Barcelona, Edifici O, Campus UAB, 08193 Bellaterra, Barcelona, Spain. E-mail: {dgeronimo, antonio, asappa}@cvc.uab.es. . T. Graf is with Volkswagen AG, Electronics Research, 38436 Wolsburg, Germany. E-mail: thorsten.graf@volkswagen.de. Manuscript received 29 July 2008; revised 31 Dec. 2008; accepted 13 May 2009; published online 21 May 2009. Recommended for acceptance by B. Schiele. For information on obtaining reprints of this article, please send e-mail to: tpami@computer.org, and reference IEEECS Log Number TPAMI-2008-07-0448. Digital Object Identifier no. 10.1109/TPAMI.2009.122. 0162-8828/10/$26.00 ß 2010 IEEE Published by the IEEE Computer Society