IEEJ TRANSACTIONS ON ELECTRICAL AND ELECTRONIC ENGINEERING IEEJ Trans 2012 Published online in Wiley Online Library (wileyonlinelibrary.com). DOI:10.1002/tee.21779 Paper Context-Based Robust Face Detection Algorithm for Surveillance Cameras Shotaro Miwa a , Non-member Hiroshi Kage, Non-member Kazuhiko Sumi, Non-member This paper describes a context-based robust face detection algorithm for surveillance cameras. Different from familiar faces captured by digital cameras, faces captured by surveillance cameras are smaller and darker with motion blurs and distortions. Furthermore, captured from top-mounted cameras, facial images are downward and partially hidden. Just using a single-face detector to detect such degraded faces is very difficult. To solve the problem, we utilize contextual information about faces of walking people. We employ a probabilistic face detection framework combining a face detector with local and global contextual information. We use a boosted fast face detector as an initial selector to pick up a small number of possible face regions in a very short time. After the fast selection of candidate face patches, as local contextual information we calculate a conditional probability in the surrounding regions using a histogram of oriented gradient (HOG) feature-based outline detector, and as global contextual information we calculate possible face patches from viewpoint information using vanishing point detection. Combining a fast boosted face detector with these contextual information, while keeping computational efficiency of the original boosted face detector, we achieved a high face detection rate of 93.7% with about 1000 times lower false-positive rate than when using a single original face detector. © 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc. Keywords: contextual information, surveillance camera, face detection, Haar-like feature, histogram of oriented gradient feature Received 8 April 2011; Revised 4 August 2011 1. Introduction This paper proposes a detection algorithm for robust and fast face detection for surveillance cameras. After the first robust and fast face detection algorithm was proposed by Viola and Jones [1], a variety of face detection algorithms have been proposed [2–4]. They try to categorize the variations of face patterns depending on different poses, or analyze internal face patterns in more detail, i.e. facial-parts-based detection. In both cases, the approach is to find better features and structures that are robust for pose and illumination changes. The above approaches are based on the traditional assumption that the inside of a face is more important and discriminative than any other regions. This assumption is almost true if the face images have no degradation, the face size is big, and variations of face patterns are restricted. But in the case of surveillance cameras, faces have huge complexity and ambiguity due to degradation. From distant and top-mounted cameras, walking people’s faces are downward, partially hidden, small, and dark. To detect such degraded faces of surveillance cameras without mistakenly detecting any non-face patterns is impossible. Although traditional detection algorithms adopt object-centered approaches using face and its inner patterns, recently there have appeared other holistic detection approaches that utilize contextual information, i.e. the relationship between a target object and its background in a whole image. These approaches have attracted the attention of researchers as a method to solve the problems of object detection in presence of ambiguity and complexity. a Correspondence to: Shotaro Miwa. E-mail: Miwa.Shotaro@bc.MitsubishiElectric.co.jp Advanced Technology R&D Center, Mitsubishi Electric Corporation, 8-1-1 Tsukaguchi-Honmachi, Amagasaki City, Hyogo 661-8661, Japan These recent context-based detection approaches propose the general frameworks for object detection. They consider configu- ration of objects in a whole image as contextual information and use it as additional information for object detectors. Torralba’s [5–7] approach utilizes contextual information to find areas of focus of attentions and their scales. He calculates outputs of Gabor filters, estimates possible locations and scales of objects in sequence, and then evaluates his approach using a variety of images, portrait images, pictures of indoors and outdoors, etc. On the other hand, Hoiem’s approach [8,9] specializes in views on streets. It utilizes contextual information to give geometrical con- straints to object detectors. It calculates a viewpoint’s configuration and three-dimensional surface orientations which are independent of the target objects in an image, and this geometrical information in an image is used for geometrical constraints for object detectors. Our approach also uses contextual information, but has more detector-centered approach for the purpose of detecting a specific object in a short time. We design our framework by considering computational efficiency as well as detection performance of faces in a real world such as surveillance cameras. A single robust and fast face detector [1] can detect degraded faces captured by surveillance cameras, but while detecting most of the faces it produces many false positives. To decrease false positives while making use of the original face detector’s computational efficiency, we utilize local contextual information surrounding the face regions and global contextual information in a whole image. By combining a fast and robust face detector with local and global contextual information, we achieved both a high face detection rate and a low false-positive rate at the same time with a small amount of additional computation. We applied our framework to images captured by a surveillance camera and confirmed its validity. This paper is organized as follows. In Section 2 we explain contextual information found in face detection in surveillance cameras. In Section 3 we explain context-based face detection framework incorporating contextual information. In Section 4 we © 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.