A one-stage modified Tiny-YOLOv3 method for Real time Moroccan license plate recognition Abdelhak Fadili #1 , Mohamed El Aroussi *2 , Youssef Fakhri #3 # LARIT Laboratory, team Network, Telecommunication, and intelligence, University Ibn Tofail Faculty of Science, BP 242, Kenitra, Morocco 1 fadili101@gmail.com 3 yousseffakhri@yahoo.fr * SIRC/LAGES, EHTP BP 8108, Casablanca, Morocco 2 moha387@yahoo.fr Abstract—Automatic license plate recognition systems (AL- PRS) have a major significant role in supporting smart city initiatives because of their promising applications in various areas, such as road safety, traffic control, smart parking, toll management, and security. Owing to ALPRS’s importance, nu- merous methods have been developed. However, many of the present solutions are still suffering from some inefficiencies in real-world situations, generally related to many constraints. In this work, we address the problem of Moroccan car license plate detection and recognition based on the state-of-the-art YOLO deep learning framework. To this end, we have made improvements in the Tiny-YOLOv3’s network structure to make it more rigorous. We proposed a real-time end-to-end ALPRS using a one-stage approach. The network is trained using images from a collected large-scale dataset of Moroccan license plates taken from different real-world scenarios, with the addition of various data augmentation techniques, to ensure robustness and efficiency under different conditions. To improve the general- ization ability and accuracy of our system, we also combined the proposed algorithm with the transfer learning technique. Extensive experiments prove that the proposed method achieves an excellent trade-off between speed and accuracy as well as the system executes the detection /recognition process in a single phase with 98.45% of accuracy and 59.5 Frames Per Second (FPS). Also, a comparative evaluation demonstrates the effectiveness of the proposed method compared to previous state- of-the-art methods. Index Terms—Real time Automatic License Plate Recognition, Deep learning, Transfer learning, Data augmentation; Tiny- YOLOv3 I. I NTRODUCTION In a world with strong demographic growth, the number of vehicles is becoming more and more important. This growth poses several problems for the efficient management of large cities: violation of the highway code, accidents, crimes, shortage of parking spaces, traffic jams, etc. In this context, researchers have tried to propose several solutions based on the application of computer techniques to intelligent transport systems. These techniques use image processing algorithms to identify vehicles. This domain is vast, since there are several types of vehicles in a road scene: trailers, trucks, buses, cars, motorcycles, etc. On the other hand, existing research has focused mainly on the automatic detection and recognition of car license plates. Fig. 1 shows that such a system has four steps [1]: • Image acquisition: Images can be extracted from video, image collections or cameras, • License plate detection: extraction of the license plate area according to certain defined characteristics or prop- erties. • Character segmentation: extraction of numbers and char- acters from the plate. • Character recognition: recognition of each character in- dividually to read the license plate. Acquired I mage Character Segmentation License Plate Detection Character Recognition Fig. 1. Process for a complete ALPRS Such a system requires a lot of resources and a significant calculation time and also requires to obtain a very good or a perfect result before moving from one step to another [1]. In practice, an insufficient result in one step automatically leads to failure in the next steps [2]. To overcome this problem, Hsu et al [3] reduced the number of steps of an ALPRS to three: in the first, they used the technique of edge clustering to extract the license plate from the image, then based on maximally stable extreme region (MSER) detector, they proposed an efficient method to segment characters from the plate. Finally, they combined the performance of LDA and SVM to recognize characters. Also for the purpose of reducing the number of intermediate steps in an ALPRS, Kessentini et al [4] proposed a two-stage ALPRS framework. They started with the license plate separation with the YOLOv2 [5] method, then in the character recognition phase, they compared the results of two methods: R-CNN [6] and YOLOv2. Taking into consideration the state of the art that we have carried out throughout our research, it has become clear that the emergence of neural networks has largely influenced research in this area. In [7], the authors used Fast YOLO both to detect the license plate and to detect the car. Subsequently, they recognized the characters still using the same method. In the International Journal of Computer Science and Information Security (IJCSIS), Vol. 19, No. 7, July 2021 69 https://sites.google.com/site/ijcsis/ ISSN 1947-5500