IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, VOL. 8, NO. 2, JUNE 2007 321 Superresolution of License Plates in Real Traffic Videos K. V. Suresh, G. Mahesh Kumar, and A. N. Rajagopalan Abstract—In this paper, a novel method to enhance license plate numbers of moving vehicles in real traffic videos is proposed. A high-resolution image of the number plate is obtained by fusing the information derived from multiple, subpixel shifted, and noisy low-resolution observations. The image to be superresolved is modeled as a Markov random field and is estimated from the observations by a graduated nonconvexity optimization proce- dure. A discontinuity adaptive regularizer is used to preserve the edges in the reconstructed number plate for improved readability. Experimental results are given on several traffic sequences to demonstrate the robustness of the proposed method to potential errors in motion and blur estimates. The method is computation- ally efficient as all operations can be implemented locally in the image domain. Index Terms—Gibbs distribution (GD), graduated nonconvexity (GNC), intelligent transport system, license plate text, Markov random field (MRF), superresolution. I. I NTRODUCTION I NTELLIGENT transport systems (ITS) that combine elec- tronics, information, communication, and network technolo- gies are increasingly being used to address traffic problems in developed as well as developing countries [1]. One of the important goals of ITS is to decipher the identity of vehicles to enable monitoring of offenses and crimes on public routes. This is typically achieved through detection and recognition of license plates by digital image processing [2], [3]. Many algorithms use image enhancement as a preprocessing step for extraction of license plates. Histogram equalization followed by contrast enhancement with a sigmoid transform function is proposed in [4]. Comelli et al. [5] apply filtering operations and histogram stretching, while Duan et al. [6] use graying, intensity normalization, and histogram equalization to improve the detectability of license plates. If a low-resolution (LR) video surveillance system captures an untoward incident on the road, a postfacto analysis of the stored video may be re- quired. However, due to image degradation, information about the identity of the vehicles involved in the incident may not be easily derivable. The problem addressed in this paper is Manuscript received June 6, 2006; revised August 31, 2006, December 18, 2006, and January 10, 2007. This work was supported by the Inter Disciplinary Research Project under ELE0304106IDRPBRAM and IIT Madras. This paper was recommended by Dr. A. Broggi. The Associate Editor for this paper was M. Trivedi. K. V. Suresh and A. N. Rajagopalan are with the Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai 600 036, TN, India (e-mail: sureshkvsit@yahoo.com; raju@ee.iitm.ac.in). G. Mahesh Kumar is with Honeywell Technology Solutions Research Labo- ratories, Bangalore 560 076, India (e-mail: maheshg_iitm@yahoo.co.in). Digital Object Identifier 10.1109/TITS.2007.895291 the improvement of the readability of license plate text using image-processing techniques. In a previous work [7], a method is suggested that enhances only the character pixels while deemphasizing the background pixels. Kang et al. [8] use a spatially adaptive regularized iterative image-restoration algorithm to remove motion blur. Cui and Huang [9] have presented a multiframe-based binariza- tion scheme for the extraction and enhancement of characters in license plates. In [10], a bilinear interpolation scheme to enhance license plate text is described. Sato et al. [11] present a subpixel interpolation-based video text enhancement scheme which assumes that video text, like news captions, is static, whereas the background moves. However, the method does not work when the text also moves. Also, interpolation cannot restore the high-frequency components lost during sampling. The video quality degrades due to various reasons such as motion blur, distance from the camera, and noise. Cost con- siderations also dictate the resolution of surveillance cameras. For the problem at hand, since there is relative motion between the camera and the vehicle, each of the LR observations rep- resents a different sampling of the scene. One can use this subpixel motion information to superresolve the license plate text. Superresolution is a process in which a high-resolution (HR) image is constructed from a set of subpixel-shifted LR images. Fundamentally, the task involves dealiasing and de- blurring [12]–[18]. It works by combining the complementary information contained in each of the samplings. In [19], a Bayesian superresolution algorithm based on the simultaneous autoregressive model developed for text image sequences is used to enhance license plates. In [20], a method for generating an HR slow-motion sequence from compressed video is sug- gested, in which an area of interest such as the license plate is slowed down and superresolved. Miravet and Rodriguez [21] use neural networks to perform superresolution of license plates. A learning-based framework has been proposed in [22] to zoom the digits in a license plate. Enhancing license plate text in real traffic videos is a chal- lenging problem for several reasons. The distance of the camera to the vehicle is typically large, rendering it difficult for even humans to decipher the text. The LR images are quite noisy and blurred. Motion and blur estimates derived from such degraded images will not be correct. It is a well-known fact that the performance of superresolution algorithms is good only when these parameters are known accurately [23]. In this paper, we present a robust superresolution algorithm for enhancement of license plate text. Following other works [24], we assume that the license plate region has been cropped by a text detection process. The HR license plate image is modeled as a Markov 1524-9050/$25.00 © 2007 IEEE