VEHICLE NUMBER RECOGNITION SYSTEM FOR AUTOMATIC TOLL TAX COLLECTION

Shoaib Rehman Soomro Mohammad Arslan Javed Fahad Ahmed Memon TE-08’, Electrical (Telecommunication) Engineering Department, Sukkur IBA

Abstract — Vehicle Number Recognition (VNR) is an image processing technology which uses efficient algorithms to detect the vehicle number from real time images. The objective is to design an efficient Vehicle Number Recognition System & to implement it for automatic toll tax collection. The system detects the vehicle first and then captures the image of the front view of the vehicle. Vehicle number plate is localized & characters are segmented. The system is designed for grayscale images so it detects the number plate regardless of its color. Template matching technique is used for character recognition. The resulting vehicle number is then compared with the available database of all the vehicles so as to come up with information about the vehicle type & to charge toll tax accordingly. The system is then allowed to open road barrier for the vehicle & generate toll tax receipt. The vehicle information (such as passing time, date, toll amount) is also stored in the database to maintain the record. The hardware & software integrated system is implemented & a working prototype model is developed. Experiments show that system successfully detects and recognize the vehicle number plate of real images of Pakistani vehicles.

Keywords: Vehicle Number Recognition; Automatic Toll Tax Collection; AVNR; VNI; ANPR

.

I. INTRODUCTION Vehicle Number Recognition (VNR) also known as Automatic Number Plate Recognition (ANPR) was invented in 1976. Many scientist groups took interest in VNR after 1990s with the development of digital camera and the increase in processing speed. VNR is an image processing technology which enables to extract vehicle license number form digital images. It consists of a still or video camera which takes the image of vehicle, find the location of the number in the image and then segments the characters & by using the template matching scheme, it translates the license number of pixel value into numerical or string. VNR can be used in many areas from speed enforcement and motorways to automation of parking lots, etc [1].It can also be used on highways & motorways to automate the toll tax collections. The system proposed through this work is efficient for automatic toll tax collection using Vehicle Number Recognition System. The earlier methods use plate color information which can detect only single color number plates or use specific color search algorithm which is computationally expensive or use artificial neural network which involves complex mathematics [2][3][4]. The proposed VNR system is efficient & color independent so that it can run real time using normal desktop PC and can recognize various standard number plates such as Sindh (Yellow), Punjab (Green & white), Government (Green) & Islamabad (white) under acceptable lighting conditions. The

proposed system consists of five steps. (1) Detection of the vehicle & capturing the image of front view of vehicle. (2) Extract & localizing number plate area using vertical edging. (3) Number plate segmentation & character separation. (4) Template matching using correlation to convert characters of pixel value to alphanumeric value. (5) Using detected license number to charge toll tax accordingly, storing details in database, receipt generating & communicating with hardware to automate road barriers. This research work is ordered as follow; section 2 reviews the literature of previous research, section 3 presents the proposed VNR system, Section 4 briefly discusses the hardware & database part to make an efficient automated toll tax collection system, Sections 5 and 6 describe experimental results, conclusion and future work respectively.

II. LITERATURE REVIEW Typical VNR System consists of four modules: image acquisition, license plate extraction, character segmentation, and character recognition. The efficiency & accuracy of the system largely depends on the second module & various approaches have been used for this purpose. There are several common searching algorithms to locate vehicle license plate. Searching algorithm rely on color information [2]. In this method a color search algorithm is used to extract the likelihood ROI in an image [2]. These algorithms are usually fast but can detect only single colored standardized number plate. High license plate extraction rate is achieved in [5], [6] based on vertical edging and mathematical morphology operations; because of having vertical edges in English characters & digits, they can be easily classified. Several algorithms also utilize neural networks for license plate extraction [4]. There are also some algorithms designed for recognizing the number plates of Pakistani vehicles [2][5]. The system [2] utilizes color searching algorithms and effectively detects the number plates of Sindh only. The other systems [5] rely on the width to height ratio of the standard number plate & matches input vertical edges with that ratio for extracting number plates. Presently, there are several common algorithms for the segmentation of license plate characters, such as segmentation through dilation, template matching & projection analysis. In the segmentation through dilation, characters of number plate are dilated vertically for separating each character & smearing algorithm is used for finding character region [7], the license plate characters are also segmented by drawing vertical

978-1-4673-4886-7/12/$31.00 ©2012 IEEE 125

projection of number plate & finding thecharacter [8]. This algorithm is simple andcreate problem if the plate have dots or image

III. PROPOSED VNR SYSTEM

Proposed VNR system has four main moduFigure 1. In the first module, it gets inpuaverage digital camera. In second module,localized using vertical edging technique. In characters of extracted plate are separatedsegmentation. Finally characters & numberusing template matching. Each module processing steps, a flow chart for the proposhown in Figure 2.

A. Image Acquisition First of all, an image is captured from the using a digital camera. In the present systemresults, images are taken using a Nokia phonresolution 300x400 pixels & for the prototypeis used. The captured image is then acquired converted into grayscale. Figure 3 shows the& converted in grayscale.

B. Vertical Edging This step explores the property of EnglisDigits, because of the digits & characters, sharp edges in the number plate area [6]. Tbeen used for extracting & locating numimage. Several methods have been prodetection. But research work shows that Sobeperformance compared with others [6]. The Stwo masks: horizontal mask and vertical m

Image Acquisition

Number Plate Extraction

Character Segmentation

Template Matching

Figure 1. Modules of VNR System Figure 2. Propo

e region of each d rapid, but can

e is noisy.

M ules as shown in ut image through , license plate is the third module,

d using character rs are indentified contains several

osed algorithm is

front of vehicle m, for simulation ne (5130) having e model, webcam in Matlab & then e image captured

sh Characters & image will have

This property has mber plate within

posed for edge el mask has good Sobel method has mask. In present

work only vertical mask of Sobel isvery much skewed, vertical edge results [5]. The result of edge detect

Figure 3. Grayscale Image Figur

C. Histogram Analysis of Bit Den

In this step, bit density of rows in fhistogram analysis. For this purpcalculated for each row & prprojection. Vertical projection is Vertical axis shows the rows of the depicts the number of white pixeshows the vertical projection of the e

Figure 5.Vertical Project

Figure-5 shows that the number platthe vertical projection. Therefore, trows with the a% highest values in are the candidate regions for the applied on the results obtained fromthe resulted image.

Figure 6. Result of Bit D

D. Row Deviation Headlights & background of the vhave vertical edges as well; thereforas candidate regions in the previou

sed Algorithm Flow

s used. If the image is not detection provides good

ion is depicted in figure 4.

re 4. Image with vertical edges

nsity figure-4 is examined using pose, number of ones is esented through vertical a graph with two axes. image, and horizontal axis ls in each row. Figure 5 edged image.

ion of bit density

te has the highest values in the next step is to find the the projection. These rows plate. Next steps will be

m this step. Figure 6 shows

Density Analysis

vehicle (such as trees) can re they can also be selected us step. Row deviation is

978-1-4673-4886-7/12/$31.00 ©2012 IEEE 126

found to prevent this. In Pakistan, number plocated in middle or offset on the vehicle’sNumber plate has vertical edges in a narrowbackground & headlights have vertical edges This feature is used in row deviation [6]. Bdeviation quantity in each row of the verticaeasily classified. An easy method for doing tnumber of deviations between “ones” and row within it 80% middle columns of figure-b% minimum deviation means this row belother rows from candidate regions will bNumber plate has vertical edges very close, the plate region will have consequent points “ones” or “zeros” and have minimum devdepicts the vertical projection of row deviatio

Figure 7. Row Deviation Analysi

As already shown & discussed, structure of thconsidered as candidate region. This is dstructure & background of the vehicle. The firesult obtained after considering deviation ana

Figure 8. Result of Row Deviatio

E. Dilation In this step, morphological operator is useimage of vertical edges is dilated horizontalland dilated vertically in second attempt. elements of dilations are 6-pixel horizontal The image is dilated to connect characters oskeleton of the number plate. Result of dilafigure 9.

plates are usually s bumper. Also a

w range, but those in an wide range.

By calculating the al edges, it can be this is to find the “zeros” for each -6. If the row has longs to plate, so be removed [6]. therefore rows in which are totally

viation. Figure 7 n.

is

he vehicles is also due to boundary igure 8 shows the alysis.

on

ed. The resulting ly in first attempt

The structuring or vertical lines. f plate & making ation is shown in

Figure 9. Dilate

F. Removing Small Objects & MeAlong with Number plate there cacandidate region. In this step those sAll those regions which have pixremoved. In this way there will beimage which is the location of numb1x15 median filter is applied on thskeleton of the number plate. The resmall objects & median filtering is s

Figure 10. After

G. Number Plate Extraction The image in figure 10 is multiplifigure 3 & then by applying horiznumber plate is cropped & extracted

H. Plate Segmentation In this step, the characters & digits and each is saved as different imanumber plate is resized to a specproperties are found and details o(character) within extracted image aout the properties, start & ending coare found & each character is saved of matrices. Figure 12 shows the nusegmentation.

Figure 11. Extracted Number Plate Fig

d Image

edian Filtering an have small lines in the small regions are removed. els less then P-pixels are

e only single object in the ber plate. In the next step, a he image to make smooth esult image after removing shown in figure-9.

Filtering

ed with original image of zontal & vertical scanning d.

of the plate are segmented age. To do this 1st of all

cific size & image region of each connected object are determined. By finding ordinates of each character as different image in a cell

umber plate characters after

gure 12. Segmented Number Plate

978-1-4673-4886-7/12/$31.00 ©2012 IEEE 127

I. Template Matching The last step of vehicle number recognition is the template matching. For matching the characters with stored characters, input images must be equal sized with the stored characters. In the present work 50x30 pixel characters are used. When the extracted characters from plate and stored characters are both equal sized & each input character image is compared with the ones already stored using cross correlation and the best similarity is measured. In the Pakistani license plates, all 36 alphanumeric characters (26 alphabets and 10 numerals) are used, therefore this system is used and each input character is correlated with all 36 alphanumeric characters. In the template matching each input character of the plate is correlated with all templates, then from the templates that character is selected which has highest value of correlation coefficient with input character.

IV. INTERFACING WITH HARDWARE MODEL The VNR system is interfaced with hardware model & database to make an automated toll tax collection system. The hardware model consists of proximity sensor to detect the presence of vehicle, a web camera to capture the image, motors to open/close the road barriers of toll plaza, desktop computer on which VNR algorithm is executed, LCD & seven segments display & a microcontroller for controlling all the components of hardware model. As the vehicle arrives at toll plaza, the inductive proximity sensor detects the vehicle and gives a signal to the PC using parallel port. The camera connected to the PC captures the image of front view of the vehicle & applies VNR algorithm on the image to recognize the vehicle’s license number. This number is then used to charge toll tax & generate receipt containing all the information of vehicle. Also, all the information such as time, date, plate number & toll amount is stored in database to maintain the record. PC then sends the signal to microcontroller using parallel port & the road barrier is opened for a time by driving motors & “Please Move Ahead” is displayed on the LCD to guide the vehicles. Complete hardware design of the system is shown in figure 13.

V. EXPERIMENTAL RESULTS Better performance of the system can be achieved when percentage values for a, b & P are set according to the quality of camera & resolution of images. After the system has been

developed, the whole system is setup for testing. The proposed algorithm has been designed in Matlab R2007 for recognition of vehicles’ number plates. The system had utilized Compaq Evo N610c with 2.00 GHz processor & 512 MB RAM. The usefulness of the proposed algorithm has been tested over images captured in various lighting conditions with a 2 megapixel camera of mobile phone (Nokia 5130). The image resolution is used as 400x300 pixels for real scenes & for prototype model a web camera is used to capture real time images with resolution of 640x480 pixels. The system takes an average of 3 to 4 seconds from detection of the vehicle to opening road barrier.

VI. CONCLUSIONS & FUTURE WORKS In this paper, we proposed a real time and efficient

method for Vehicle Number recognition & implementation of that method for automatic toll tax collection. The system has been tested on many images of various lighting conditions & system can be implemented on motorways & highways for automatic toll tax collection.

The proposed system works quite well however, there are

still areas for improvement. The camera used in this project is average quality & cannot detect fast changing targets due to the long shutter time. The system robustness and speed can be increased if high resolution camera is used. The template matching method used in this project for the recognition is subject to problems when detecting the characters such as 8 & B or 0 & O. The frequency transformation can be used while correlating to improve the number recognition of the system.

.

REFERENCES [1] The Automatic Number Plate Recognition Tutorial, http://www.anpr-

tutorial.com, Accessed on May-2012.

[2] M. Tahir Qadri, M. Asif “Automatic Number Plate Recognition System for Vehicle Identification using OCR,” International Conference on Education Technology and Computer, pp 335 – 338, 2009.

[3] V. Swetha, D.R. Sandeep “Automatic Authorized Vehicle Recognition System,” Chennai and Dr.MGR University Second International Conference on Sustainable Energy and Intelligent System (SEISCON), pp 789 – 790, 2011.

[4] V. Koval, V. Turchenko, V. Kochan, A. Sachenko, G. Markowsky “Smart License Plate Recognition System Based on Image Processing Using Neural Network,” IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, pp 123 – 127, 2003.

[5] A. Tahir, H. Adnan Habib, M. Fahad Khan “License Plate Recognition Algorithm for Pakistani License Plates,” Canadian Journal on Image Processing and Computer Vision Vol. 1, No. 2, pp 30-36, April 2010.

[6] F. Faradji, A. Hossein Rezaie, M. Ziaratban “A Morphological Based License Plate Locating System,” IEEE International Conference on Image Processing(ICIP), pp 57-60, 2007.

[7] S. Ozbay, and E. Ercelebi “Automatic Vehicle Identification by Plate Recognition” World Academy of Science, Engineering and Technology 9, pp 222-225, 2005.

Figure 13. System Hardware Design

Receipt Printer

Matlab Executing VNR Algorithm

Camera Microcontroller

Proximity Sensor Road

Barriers LCD & 7-Segment

DATABASE

978-1-4673-4886-7/12/$31.00 ©2012 IEEE 128

[8] Ch. Lakshmi, A.J Rani, K.S Ramakrishna M.Kanti Kiran “A Novel

Approach for Indian License Plate Recognition System,” International Journal of Advanced Engineering Science and Technologies (IJAEST) Vol No. 6, Issue No. 1, pp 10-14, 2010.

978-1-4673-4886-7/12/$31.00 ©2012 IEEE 129