A Near-Infrared Fusion Scheme for Automatic Detection of Vehicle Passengers I. Pavlidis P. Symosek B. Fritz N. Papanikolopoulos Honeywell Technology Center University of Minnesota 3660 Technology Dr. Dept. of Computer Science MN65-2500 200 Union St. S.E. Minneapolis, MN 55418 Minneapolis, MN 55455 (pavlidis, symosek, fritz)@htc.honeywell.com npapas@cs.umn.edu Fax 612 951-7438 Fax 612 625-0163 Tel. 612 951-7338 Tel. 612 625-0572 Abstract We undertook a study to determine if the automatic de- tection and counting of vehicle passengers is feasible. An automated passenger counting system would greatly facil- itate the operation of freeway lanes reserved for car-pools (HOV lanes). In the present paper we report our findings regarding the appropriate sensor phenomenology and ar- rangement for the task. We propose a novel system based on fusion of near-infrared imaging signals and we demonstrate its adequacy with theoretical and experimental arguments. 1. Introduction There are compelling reasons for the existence of an au- tomatic passenger counting system in the HOV lane. In particular, such a system will be useful in the following re- spects: 1. It will facilitate the gathering of statistical data for road construction planning. The gathering of usage statis- tics in the HOV lane is mandated by the U.S. Federal Highway Administration. Currently, the gathering of data is performed manually. This is obviously labori- ous, inefficient, and prone to error. 2. It will facilitate law enforcement in the HOV lane. Currently, HOV lane enforcement requires substantial commitments of State Highway Patrol personnel and equipment. HOV lane enforcement has other costs as well. These include the risks of high-speed pursuit in To whom all correspondence should be addressed. lanes adjacent to stop-and-go traffic and the deteriora- tion of traffic flow when tickets are issued during peak commute periods. 3. It will enable the States to offer the option to single drivers to use some HOV lanes for a nominal monthly fee. A complete HOV monitoring system suitable for the above applications will consist of a passenger detector and a license plate reader. Although, substantial work has been reported to the technical literature regarding license plate readers, work for automated passenger detectors is still in its infancy. There are three major technical challenges in the development of an automatic passenger detector: 1. The imaging signal should provide a clear picture of the interior of the car. The contrast between the human silhouettes and the background should be sufficient to provide for reliable image processing. 2. The pattern recognition algorithm that performs the passenger detection should exhibit high recognition rates and robust behavior. Of course its performance, depends to a significant degree on the quality of the imaging signal. Even the best pattern recognition algo- rithm cannot perform reliably when the imaging signal is corrupted with noise. 3. The system architecture should be designed in such a way that will ensure real-time operation, passenger de- tection in both the front and back seats, and protection from the weather elements. In this paper we address the first from the above three technical challenges. We describe a novel near-infrared fu- sion system that provides high quality imaging signal both