Abstract—This paper presents an approach for evaluating video-based enforcement systems for motorway toll collection, which has been applied to the case of Portugal’s largest motorway operator, Brisa. The results of this evaluation have contributed to the design of a new LPR system, denoted advanced license plate recognition (ALPR), also described in this paper. The ALPR is currently being deployed not only by Brisa, but also by other operators that use the Via Verde system. A significant decrease in the need for human intervention has been an important improvement, in which the introduction of a tunable and trustable confidence level in the LPR process has played a key part. I. INTRODUCTION IDEO-based enforcement systems in motorways have become increasingly widespread, due to the fact that they allow efficient data capture 24 hours a day, as well as the easy storage, handling and transmission of the data in digital form [1,2,3]. Many of the subsequent enforcement procedures, based on License Plate Recognition (LPR) technology, can be automated, reducing operating costs and enabling greater effectiveness [4,5,6]. The role of such systems ranges from detecting and documenting traffic violations to providing enforcement in the electronic toll collection. The focus of the present paper is on the latter role, for the case of the Via Verde toll collection system. This work is integrated in the wider scope of the Intelligent Transport System - Interoperability Bus (ITS-Ibus), which is undergoing development [7]. The Via Verde was introduced in 1991 (which makes it a pioneering electronic toll collection system worldwide), on Manuscript received March 13, 2006. This work was supported in part by Brisa Auto-Estradas de Portugal, SA. J. G. Silva is with the Multimedia and Machine Learning Group, DEETC, ISEL, R. Conselheiro Emídio Navarro, 1950-062 Lisbon Portugal ( e-mail: jgs@ isel.ipl.pt ). G. C. Marques is with the Multimedia and Machine Learning Group, DEETC, ISEL, R. Conselheiro Emídio Navarro, 1950-062 Lisbon Portugal ( e-mail: gmarques@ deetc.isel.ipl.pt ). P. M. Jorge is with the Multimedia and Machine Learning Group, DEETC, ISEL, R. Conselheiro Emídio Navarro, 1950-062 Lisbon Portugal ( e-mail: pmj@ deetc.isel.ipl.pt ). A. J. Abrantes is with the Multimedia and Machine Learning Group, DEETC, ISEL, R. Conselheiro Emídio Navarro, 1950-062 Lisbon Portugal (phone: +351-21-8317251; fax: +351-21-8317114; e-mail: aja@ deetc.isel.ipl.pt ).A. L. Osório is with the GIATSI Research Group, DEETC, ISEL, R. Conselheiro Emídio Navarro, 1950-062 Lisbon Portugal ( e-mail: aosorio@ deetc.isel.ipl.pt ). J. S. Gomes is with the DIT, Brisa Auto-Estradas de Portugal, SA, Quinta da Torre da Aguilha, Edifício Brisa, 2795-599 S. Domingos de Rana, Portugal ( e-mail: jorge.gomes@ brisa.pt ). J. C. Braga is with Brisa Auto-Estradas de Portugal, SA, Quinta da Torre da Aguilha, Edifício Brisa, 2795-599 S. Domingos de Rana, Portugal ( e-mail: j.braga@ brisa.pt ). motorways operated by Brisa Auto-Estradas de Portugal, having subsequently expanded its coverage to other motorway operators and to other applications, e. g. controlling access to parking lots. From the inception it was understood that, as a gateless system, the Via Verde would require enforcement in order to be successful. In regular conditions, toll collection is performed through Dedicated Short Range Communication (DSRC) at microwave frequencies (5.8 GHz) between antennas at the toll plaza and a tag, also called On-Board Unit (OBU), fixed to the windshield of the passing vehicle. In the event of an irregularity, for example a vehicle without an OBU or a communication failure, a rear photograph of the vehicle is taken and forwarded to the toll clearing company, Via Verde Portugal (VVP), for further processing. From the photograph, it is possible to obtain the license plate number and match it to the client database, pursuing legal action if necessary, or simply initiating a normal toll charge in case the transaction failure was due to technical reasons (e. g. low battery on the OBU). Given the number of photographs per day (around 15,000), manually reading the license plate numbers from all photographs is not a feasible option. In order not to remain limited to processing a small sample of the violations, and since in any case manually reading license plates is a tedious and time-consuming (and therefore expensive) process, a semi-automatic procedure based on optical license plate recognition (LPR) technology was adopted. The current system, based on rear capture, has been in operation for approximately three years. The procedure is not fully automatic because currently available LPR methods still have a significant error rate. It should be noted that even an error rate of 5%, corresponding to 95% correctly recognized license plates, might lead to thousands of fines being wrongly issued per year, which would seriously undermine the system’s credibility. Also, there are specific difficulties regarding trucks, trailers and vehicles with logotypes and written signs, which are often mistaken for license plates by LPR systems. Even when manually processing the photograph, in such cases (which amount from 3% to 5% of the total number of pictures) it often happens that the visible, rear plate belongs to the trailer instead of the vehicle itself, rendering identification of the vehicle owner considerably more difficult. Evaluation of an LPR-Based Toll Enforcement System on Portuguese Motorways Jorge G. Silva, Gonçalo C. Marques, Pedro M. Jorge, Arnaldo J. Abrantes, António L. Osório, Jorge S. Gomes and José C. Braga V