Multi-scale Structural Saliency for Signature Detection Guangyu Zhu 1 , Yefeng Zheng 2 , David Doermann 1 , and Stefan Jaeger 1 1 Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 2 Siemens Corporate Research, 755 College Road East, Princeton, NJ 08540 Abstract Detecting and segmenting free-form objects from clut- tered backgrounds is a challenging problem in computer vi- sion. Signature detection in document images is one classic example and as of yet no reasonable solutions have been presented. In this paper, we propose a novel multi-scale ap- proach to jointly detecting and segmenting signatures from documents with diverse layouts and complex backgrounds. Rather than focusing on local features that typically have large variations, our approach aims to capture the struc- tural saliency of a signature by searching over multiple scales. This detection framework is general and computa- tionally tractable. We present a saliency measure based on a signature production model that effectively quantifies the dynamic curvature of 2-D contour fragments. Our evalua- tion using large real world collections of handwritten and machine printed documents demonstrates the effectiveness of this joint detection and segmentation approach. 1. Introduction Detecting free-form objects pose fundamental chal- lenges in a number of aspects. First, detection needs to be robust in the presence of cluttered backgrounds. Sec- ond, non-rigid objects can have very large intra-class vari- ations, making it almost impossible to model without over- fitting the data. Third, the contours of many such complex objects are fragmented 2-D signals, so reliably recovering the ordering of points along contour fragments from off- line images is difficult in general. In addition, recognition and retrieval require well segmented objects from the de- tected regions, to minimize the effects of outliers during matching. Detecting signatures from documents is an ex- ample of one such difficult problem in which diverse layout structures, complex background, and noise make contour- based learning hard. Furthermore, the foreground content of documents generally includes a mixture of machine printed text, handwriting, diagrams, and other elements. Handwrit- ten signature detection and segmentation is still an open re- search area and to our best knowledge, no comprehensive solutions have been presented in the literature. As signatures are a pervasive method of individual iden- tification and document authentication, they provide an im- portant form of indexing that enables exploration of large document repositories. Given a large collection of docu- ments, searching for a specific signature is a highly effec- tive way of retrieving documents authorized or authored by an individual. Such need arises frequently in the discovery phase of legal and intelligence investigations [11]. Prior research on off-line signatures has almost exclu- sively focused on signature verification and identification [18, 14, 10, 5] in the context of biometrics to perform au- thentication. For signature verification, the problem is to decide whether a sample signature is genuine or a forgery by comparing it with stored reference signatures. Signa- ture identification is essentially a writer identification prob- lem, whose objective is to find the author of a test signature given a database of signature exemplars from different sign- ers. Most studies published to date assume that an almost perfect detection and segmentation is available [16]. Solving the problem of signature detection and segmen- tation is pivotal for signature-based document indexing and retrieval. Equally important is that a solution will also bene- fit off-line signature verification and identification in a range of domains. In addition, the ability to robustly detect signa- tures and extract them intact from volumes of documents is needed in many business and government applications. In this paper, we propose a new multi-scale approach to detecting and extracting signatures from document images. Rather than viewing a signature as a collection of local fea- tures, we treat it as a global symbol that exhibits character- istic structural saliency. Computation in the proposed multi- scale framework for joint object detection and segmentation is carried out efficiently in a coarse-to-fine scheme on con- tour fragments. We employ a novel saliency measure based on a signature production model, which assumes two gen- eral degrees of freedom. The model enables us to capture the dynamic curvature in a signature without recovering its temporal information—a task shown to be difficult for un- constrained off-line handwriting due to structural changes [6]. The signature detection approach can also be applied to on-line handwritten notes collected on a PDA or Tablet PC, where the trajectories of the pen are readily available. 1-4244-1180-7/07/$25.00 ©2007 IEEE