design an efficient Vehicle Number Recognition System & to implement it for automatic toll tax collection. The system detects the vehicle first and then captures ...
VEHICLE NUMBER RECOGNITION SYSTEM FOR AUTOMATIC TOLL TAX COLLECTION Shoaib Rehman Soomro
Mohammad Arslan Javed
Fahad Ahmed Memon
TE-08’, Electrical (Telecommunication) Engineering Department, Sukkur IBA
Abstract — Vehicle Number Recognition (VNR) is an image
proposed system consists of five steps. (1) Detection of the vehicle & capturing the image of front view of vehicle. (2) Extract & localizing number plate area using vertical edging. (3) Number plate segmentation & character separation. (4) Template matching using correlation to convert characters of pixel value to alphanumeric value. (5) Using detected license number to charge toll tax accordingly, storing details in database, receipt generating & communicating with hardware to automate road barriers.
processing technology which uses efficient algorithms to detect the vehicle number from real time images. The objective is to design an efficient Vehicle Number Recognition System & to implement it for automatic toll tax collection. The system detects the vehicle first and then captures the image of the front view of the vehicle. Vehicle number plate is localized & characters are segmented. The system is designed for grayscale images so it detects the number plate regardless of its color. Template matching technique is used for character recognition. The resulting vehicle number is then compared with the available database of all the vehicles so as to come up with information about the vehicle type & to charge toll tax accordingly. The system is then allowed to open road barrier for the vehicle & generate toll tax receipt. The vehicle information (such as passing time, date, toll amount) is also stored in the database to maintain the record. The hardware & software integrated system is implemented & a working prototype model is developed. Experiments show that system successfully detects and recognize the vehicle number plate of real images of Pakistani vehicles.
This research work is ordered as follow; section 2 reviews the literature of previous research, section 3 presents the proposed VNR system, Section 4 briefly discusses the hardware & database part to make an efficient automated toll tax collection system, Sections 5 and 6 describe experimental results, conclusion and future work respectively. II. LITERATURE REVIEW Typical VNR System consists of four modules: image acquisition, license plate extraction, character segmentation, and character recognition. The efficiency & accuracy of the system largely depends on the second module & various approaches have been used for this purpose. There are several common searching algorithms to locate vehicle license plate. Searching algorithm rely on color information [2]. In this method a color search algorithm is used to extract the likelihood ROI in an image [2]. These algorithms are usually fast but can detect only single colored standardized number plate. High license plate extraction rate is achieved in [5], [6] based on vertical edging and mathematical morphology operations; because of having vertical edges in English characters & digits, they can be easily classified. Several algorithms also utilize neural networks for license plate extraction [4]. There are also some algorithms designed for recognizing the number plates of Pakistani vehicles [2][5]. The system [2] utilizes color searching algorithms and effectively detects the number plates of Sindh only. The other systems [5] rely on the width to height ratio of the standard number plate & matches input vertical edges with that ratio for extracting number plates. Presently, there are several common algorithms for the segmentation of license plate characters, such as segmentation through dilation, template matching & projection analysis. In the segmentation through dilation, characters of number plate are dilated vertically for separating each character & smearing algorithm is used for finding character region [7], the license plate characters are also segmented by drawing vertical
Keywords: Vehicle Number Recognition; Automatic Toll Tax Collection; AVNR; VNI; ANPR .
I. INTRODUCTION Vehicle Number Recognition (VNR) also known as Automatic Number Plate Recognition (ANPR) was invented in 1976. Many scientist groups took interest in VNR after 1990s with the development of digital camera and the increase in processing speed. VNR is an image processing technology which enables to extract vehicle license number form digital images. It consists of a still or video camera which takes the image of vehicle, find the location of the number in the image and then segments the characters & by using the template matching scheme, it translates the license number of pixel value into numerical or string. VNR can be used in many areas from speed enforcement and motorways to automation of parking lots, etc [1].It can also be used on highways & motorways to automate the toll tax collections. The system proposed through this work is efficient for automatic toll tax collection using Vehicle Number Recognition System. The earlier methods use plate color information which can detect only single color number plates or use specific color search algorithm which is computationally expensive or use artificial neural network which involves complex mathematics [2][3][4]. The proposed VNR system is efficient & color independent so that it can run real time using normal desktop PC and can recognize various standard number plates such as Sindh (Yellow), Punjab (Green & white), Government (Green) & Islamabad (white) under acceptable lighting conditions. The 978-1-4673-4886-7/12/$31.00 ©2012 IEEE
125
work only vertical mask of Sobel iss used. If the image is not very much skewed, vertical edge detection provides good results [5]. The result of edge detection is depicted in figure 4.
projection of number plate & finding thee region of each character [8]. This algorithm is simple andd rapid, but can create problem if the plate have dots or imagee is noisy. M III. PROPOSED VNR SYSTEM
Proposed VNR system has four main moduules as shown in Figure 1. In the first module, it gets inpuut image through average digital camera. In second module,, license plate is localized using vertical edging technique. In the third module, characters of extracted plate are separatedd using character segmentation. Finally characters & numberrs are indentified using template matching. Each module contains several processing steps, a flow chart for the propoosed algorithm is shown in Figure 2.
Figure 3. Grayscale Image
Figurre 4. Image with vertical edges
nsity C. Histogram Analysis of Bit Den In this step, bit density of rows in figure-4 f is examined using histogram analysis. For this purp pose, number of ones is calculated for each row & presented through vertical projection. Vertical projection is a graph with two axes. Vertical axis shows the rows of the image, and horizontal axis depicts the number of white pixels in each row. Figure 5 shows the vertical projection of the edged e image.
Image Acquisition
Number Plate Extraction
Character Segmentation Figure 5.Vertical Projection of bit density
Figure-5 shows that the number platte has the highest values in the vertical projection. Therefore, the t next step is to find the rows with the a% highest values in the projection. These rows are the candidate regions for the plate. Next steps will be m this step. Figure 6 shows applied on the results obtained from the resulted image.
Template Matching Figure 1. Modules of VNR System
Figure 2. Proposed Algorithm Flow
A. Image Acquisition First of all, an image is captured from the front of vehicle m, for simulation using a digital camera. In the present system results, images are taken using a Nokia phonne (5130) having resolution 300x400 pixels & for the prototypee model, webcam is used. The captured image is then acquired in Matlab & then converted into grayscale. Figure 3 shows thee image captured & converted in grayscale. B. Vertical Edging This step explores the property of Englissh Characters & Digits, because of the digits & characters, image will have This property has sharp edges in the number plate area [6]. T been used for extracting & locating num mber plate within image. Several methods have been proposed for edge detection. But research work shows that Sobeel mask has good performance compared with others [6]. The S Sobel method has two masks: horizontal mask and vertical m mask. In present 978-1-4673-4886-7/12/$31.00 ©2012 IEEE
Figure 6. Result of Bit Density D Analysis
D. Row Deviation v (such as trees) can Headlights & background of the vehicle have vertical edges as well; thereforre they can also be selected as candidate regions in the previou us step. Row deviation is 126
found to prevent this. In Pakistan, number pplates are usually located in middle or offset on the vehicle’ss bumper. Also a Number plate has vertical edges in a narrow w range, but those background & headlights have vertical edges in an wide range. By calculating the This feature is used in row deviation [6]. B deviation quantity in each row of the verticaal edges, it can be easily classified. An easy method for doing tthis is to find the number of deviations between “ones” and “zeros” for each row within it 80% middle columns of figure--6. If the row has b% minimum deviation means this row bellongs to plate, so other rows from candidate regions will bbe removed [6]. Number plate has vertical edges very close, therefore rows in the plate region will have consequent points which are totally “ones” or “zeros” and have minimum devviation. Figure 7 depicts the vertical projection of row deviatio n.
Figure 9. Dilated Image
F. Removing Small Objects & Meedian Filtering Along with Number plate there caan have small lines in the candidate region. In this step those small s regions are removed. All those regions which have pixels less then P-pixels are removed. In this way there will bee only single object in the image which is the location of numb ber plate. In the next step, a 1x15 median filter is applied on th he image to make smooth skeleton of the number plate. The reesult image after removing small objects & median filtering is shown s in figure-9.
Figure 7. Row Deviation Analysiis
As already shown & discussed, structure of thhe vehicles is also considered as candidate region. This is ddue to boundary structure & background of the vehicle. The fiigure 8 shows the result obtained after considering deviation anaalysis.
Figure 10. After Filtering
G. Number Plate Extraction The image in figure 10 is multiplied with original image of figure 3 & then by applying horizzontal & vertical scanning number plate is cropped & extracted d. H. Plate Segmentation In this step, the characters & digits of the plate are segmented and each is saved as different imaage. To do this 1st of all number plate is resized to a speccific size & image region properties are found and details of o each connected object (character) within extracted image are a determined. By finding out the properties, start & ending coordinates of each character are found & each character is saved as different image in a cell umber plate characters after of matrices. Figure 12 shows the nu segmentation.
Figure 8. Result of Row Deviatioon
E. Dilation In this step, morphological operator is useed. The resulting image of vertical edges is dilated horizontallly in first attempt and dilated vertically in second attempt. The structuring elements of dilations are 6-pixel horizontal or vertical lines. The image is dilated to connect characters of plate & making skeleton of the number plate. Result of dilaation is shown in figure 9.
Figure 11. Extracted Number Plate
978-1-4673-4886-7/12/$31.00 ©2012 IEEE
127
Fig gure 12. Segmented Number Plate
I. Template Matching The last step of vehicle number recognition is the template matching. For matching the characters with stored characters, input images must be equal sized with the stored characters. In the present work 50x30 pixel characters are used. When the extracted characters from plate and stored characters are both equal sized & each input character image is compared with the ones already stored using cross correlation and the best similarity is measured. In the Pakistani license plates, all 36 alphanumeric characters (26 alphabets and 10 numerals) are used, therefore this system is used and each input character is correlated with all 36 alphanumeric characters. In the template matching each input character of the plate is correlated with all templates, then from the templates that character is selected which has highest value of correlation coefficient with input character.
developed, the whole system is setup for testing. The proposed algorithm has been designed in Matlab R2007 for recognition of vehicles’ number plates. The system had utilized Compaq Evo N610c with 2.00 GHz processor & 512 MB RAM. The usefulness of the proposed algorithm has been tested over images captured in various lighting conditions with a 2 megapixel camera of mobile phone (Nokia 5130). The image resolution is used as 400x300 pixels for real scenes & for prototype model a web camera is used to capture real time images with resolution of 640x480 pixels. The system takes an average of 3 to 4 seconds from detection of the vehicle to opening road barrier.
VI. CONCLUSIONS & FUTURE WORKS In this paper, we proposed a real time and efficient method for Vehicle Number recognition & implementation of that method for automatic toll tax collection. The system has been tested on many images of various lighting conditions & system can be implemented on motorways & highways for automatic toll tax collection.
IV. INTERFACING WITH HARDWARE MODEL The VNR system is interfaced with hardware model & database to make an automated toll tax collection system. The hardware model consists of proximity sensor to detect the presence of vehicle, a web camera to capture the image, motors to open/close the road barriers of toll plaza, desktop computer on which VNR algorithm is executed, LCD & seven segments display & a microcontroller for controlling all the components of hardware model. As the vehicle arrives at toll plaza, the inductive proximity sensor detects the vehicle and gives a signal to the PC using parallel port. The camera connected to the PC captures the image of front view of the vehicle & applies VNR algorithm on the image to recognize the vehicle’s license number. This number is then used to charge toll tax & generate receipt containing all the information of vehicle. Also, all the information such as time, date, plate number & toll amount is stored in database to maintain the record. PC then sends the signal to microcontroller using parallel port & the road barrier is opened for a time by driving motors & “Please Move Ahead” is displayed on the LCD to guide the vehicles. Complete hardware design of the system is shown in figure 13. D A T A B A S
E
The proposed system works quite well however, there are still areas for improvement. The camera used in this project is average quality & cannot detect fast changing targets due to the long shutter time. The system robustness and speed can be increased if high resolution camera is used. The template matching method used in this project for the recognition is subject to problems when detecting the characters such as 8 & B or 0 & O. The frequency transformation can be used while correlating to improve the number recognition of the system. . REFERENCES [1]
The Automatic Number Plate Recognition Tutorial, http://www.anprtutorial.com, Accessed on May-2012.
[2]
M. Tahir Qadri, M. Asif “Automatic Number Plate Recognition System for Vehicle Identification using OCR,” International Conference on Education Technology and Computer, pp 335 – 338, 2009.
[3]
V. Swetha, D.R. Sandeep “Automatic Authorized Vehicle Recognition System,” Chennai and Dr.MGR University Second International Conference on Sustainable Energy and Intelligent System (SEISCON), pp 789 – 790, 2011.
[4]
V. Koval, V. Turchenko, V. Kochan, A. Sachenko, G. Markowsky “Smart License Plate Recognition System Based on Image Processing Using Neural Network,” IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, pp 123 – 127, 2003.
[5]
A. Tahir, H. Adnan Habib, M. Fahad Khan “License Plate Recognition Algorithm for Pakistani License Plates,” Canadian Journal on Image Processing and Computer Vision Vol. 1, No. 2, pp 30-36, April 2010.
[6]
F. Faradji, A. Hossein Rezaie, M. Ziaratban “A Morphological Based License Plate Locating System,” IEEE International Conference on Image Processing(ICIP), pp 57-60, 2007.
[7]
S. Ozbay, and E. Ercelebi “Automatic Vehicle Identification by Plate Recognition” World Academy of Science, Engineering and Technology 9, pp 222-225, 2005.
Matlab Executing VNR Algorithm
Microcontroller
Camera
Receipt Printer
Proximity Sensor Road Barriers
LCD & 7Segment
Figure 13. System Hardware Design
V. EXPERIMENTAL RESULTS Better performance of the system can be achieved when percentage values for a, b & P are set according to the quality of camera & resolution of images. After the system has been 978-1-4673-4886-7/12/$31.00 ©2012 IEEE
128
[8]
Ch. Lakshmi, A.J Rani, K.S Ramakrishna M.Kanti Kiran “A Novel Approach for Indian License Plate Recognition System,” International Journal of Advanced Engineering Science and Technologies (IJAEST) Vol No. 6, Issue No. 1, pp 10-14, 2010.
978-1-4673-4886-7/12/$31.00 ©2012 IEEE
129
