A Modular System to Recognize Numerical Amounts on Brazilian Bank Cheques L. S. Oliveira , R. Sabourin , F. Bortolozzi and C. Y. Suen PUCPR Pontif´ ıcia Universidade Cat´ olica do Paran´ a (PPGIA-LARDOC) Rua Imaculada Conceic ¸˜ ao 1155, 80215-901 - Curitiba, PR - BRAZIL soares,fborto @ppgia.pucpr.br ETS - Ecole de Technologie Sup´ erieure (LIVIA) 1100, rue Notre Dame Ouest, Montreal, H3C 1K3, CANADA sabourin@gpa.etsmtl.ca CENPARMI Centre for Pattern Recognition and Machine Intelligence 1455 de Maisonneuve Blvd. West, Suite GM 606 - Montreal, H3G 1M8, CANADA suen@cenparmi.concordia.ca Abstract This paper presents a modular system to recognize nu- merical amounts on Brazilian bank cheques. The system uses a segmentation-based recognition approach and the recognition function is based on a Recognition and Veri- fication strategy. Our approach consists of combining the outputs from different levels such as segmentation, recogni- tion and post-processing in a probabilistic model. A new feature set is introduced to the verifier module in order to detect segmentation effects such as over-segmentation and under-segmentation. Finally, we present experimental re- sults on two databases: numerical amounts and NIST SD19. The latter aims at validating the concept of modular system and showing the robustness of the system over a well-known database. 1 Introduction Automatic processing of bank cheques has been a very popular task for handwriting recognition research. This is motivated by the availability of relatively inexpensive CPU power, and the possibility to reduce considerably the man- ual effort involved in this task. Comparated to a zip-code recognition problem, bank cheque systems have to take into account the large variability in the representation of a nu- merical amount, e.g., the number of components to be iden- tified. We present in this paper a modular recognition sys- tem for handwritten numerical amounts on Brazilian bank cheques. This system takes a segmentation-based recogni- tion approach where an explicit segmentation algorithm de- termines the cut regions and provides a multiple spatial rep- resentation. This kind of system has to solve a crucial prob- lem: distinguishing, at the recognition stage, a sequence corresponding to an inter-character segmentation from an- other relative to an intra-character segmentation. In order to deal with this problem we have used a strategy based on Recognition and Verification where the integration of all modules is done through a probabilistic model, which is in- spired by information theory models [5]. The main contribution of this work is related to the ver- ification module. We propose a new feature set, which takes into account multi-level concavity analysis and con- textual information, in order to feed the verifier module. We present also the concept of modular recognition system, and we show how such a recognition system can deal with dif- ferent applications. In order to validate such a concept and to evaluate the robustness of the system, we present some experiments on NIST SD19. 2 Probabilistic Model and Modular System The goal of the probabilistic model is to define a func- tion that combines all the system modules in order to allow a sound integration of all knowledge sources used to infer a plausible interpretation. The probabilistic model that we are using has been applied to speech recognition [6], hand- written word recognition [9] and handwritten digit recog- nition [3]. Such a model estimates the most probable in- terpretation of the written amount (noted ). Its input corresponds to an image after pre-processing and segmen- tation which provides a list of elementary sub-images. In a