License Plate Character Segmentation and Recognition Based on RBF Neural Network

Baoming shan College of Automation and Electronic Engineering

Qingdao University of Science and Technology Qingdao, 266042, China baoming_shan@163.com

Abstract—Character segmentation and recognition is the research hotspot of vehicle license plate recognition technology. A new method is presented in this paper. Based on the vehicle license location, the segment method of vertical projection information with prior knowledge is used to slit characters and extract the statistical features. Then the RBF neural network is used to recognize characters with the feature vector as input. The results show that this method can recognize characters precisely and improve the ability of license plate character recognition effectively.

Keywords: License plate character recognition; vertical projection; feature extractiont; RBF neural network

I. INTRODUCTION

The vehicle license plate recognition technology has important application in Intelligent Transportation Systems (ITS) and has been used extensively in highway and bridge charge, port, airport gate monitoring, and so on. The key technologies of license plate recognition are license location and character recognition. At present domestic and foreign scholars have made a lot of researches on them and promoted the development in this domain [1][2][3]. In this article character recognition is mainly studied.

Usually the license plate recognition system is running out of door, the images are easily interrupted because they are taken in complex background. It is a hot problem that how to segment character from images of different backgrounds precisely and reliably. At present there are many kinds of algorithms of license plate character recognition [4][5], the typical method is based on pattern matching[6]. But the pattern matching is sensitive to those images of expansion, inclined and background disturbed, and is not ideal. There are several other mature algorithms such as statistical pattern recognition, artificial neural network method, etc. Artificial neural network method has many advantages [7]. It has large parallel processing ability, good adaptability, strong study ability, strong association function and fault tolerance function. So it has high recognition rate, strong anti-disturbance ability and flexible, but the study process is long. In this paper based on the pre-processing images, the one-dimension vertical projection are used to slit characters, then extract the statistical features, make the feature vector as input and apply the RBF neural network to recognize characters to improve the ability of license plate character recognition effectively.

II. LICENSE PLATE CHARACTER SEGMENTATION

To recognize the Chinese characters, letters and numbers, the single character must be extracted from vehicle plates. The segmentation results directly affect character recognition. It is binarized before separation, then row scanning method based on prior knowledge is applied to process the frame and rivet. In this article the segment method of vertical projection information with prior knowledge is proposed, and is modified to improve the precision.

The standard license plate consists of 7 characters which are arranged regularly. If the total gray values of every column pixels are calculated based on binary images, the vertical projection can be acquired. The vertical projection chart is shown in Fig.1.

Figure 1. The vertical projection of license plate

In Fig.1 the X axis is vehicle plate images’ column, the Y axis is the projection value. The figure shows that the vertical projection of plate has 7 groups of regular comb chart, there is a fixed space size between groups, except the space between the 2nd and 3rd is bigger. Significantly every comb chart stands for one character, and the space between groups is corresponding to the space between characters. In vertical projection of binary images as long as the space position is found, the every character position is determined. While one of the important conditions to judge space is that the projection value is 0 in space position.

The size of standard vehicle license is 45 90 ,character space is 12 , character space between the 2

mmmm nd and

3rd is 34 , the total width is 409 . Using these prior knowledge, the character size of factual plate image can be calculated, where W is the total width.

mm mm

The character width:

2010 Second International Workshop on Education Technology and Computer Science

978-0-7695-3987-4/10 $26.00 © 2010 IEEE

DOI 10.1109/ETCS.2010.464

86

40945WcWidth

The usual space between characters:

40912Wspace

The space between the 2nd and 3rd :

342409

space W

So the start position to search ( ) can be calculated as follows:

spos

2212 spacespacecWidthspos

To make the character segmentation algorithm practical standard, several questions must be solved such as the images in case of adhesion, deviation of right or left and disturbance by minor noise.

1) The character width and space can be calculated by (1) (3) . It can solve the segmentation of adhesion character using prior knowledge.

For example, segment characters by the way of searching to right, the idea is as follows:

First search the start position sC and end position eC ofcharacter ( n =2, 3, )n

Then judge whether the ( ) value is more than ;

eC - sCcWidth

If ( eC ) , that means there is no adhesion in character ;

- sC cWidthn

If ( )> , that means n is adhesion character. It must be segmented compulsory. So-called compulsory segmentation, it means that let

eC - sC cWidth

sC be the left bound of character , that is to say,

.Then the right bound is n

cpos[n]left = sC + cWidth

sC + , left bound of character n+ is , that is

. Last letbe , back to the 2

cWidth 1cpos[n]right = cpos[n]right +space

[ 1] [ ]cpos n left cpos n right spacesC = cpos[n]left nd step to judge ( eC sC )again until it fits to the standard width.

2) To avoid the left and right deviation, it is necessary to define the actual plate width. In another word, to search the plate’s left and right bound in binary images.

By the use of vertical projection, search from left to right until find the column j that the vertical projection is more

than a certain threshold T (T is empirical value), and make it as start column

-jP

sC ; then, search from right to left until find the column j that the vertical projection is more than

threshold , and make it as the end column . Last calculate the license plate width by W = again. Now substitute the afresh plate width into (1) ~ (3) to solve the character segmentation failure because of left and right deviation.

-jP

T eCeC - sC

3) The minor noise includes the disturbance of points between the 2nd and 3rd character and other glitch as well.

In character segmentation algorithm use the condition of 0kP to judge character position. We can set a noise

threshold of fault tolerance no , and change judgment condition to

iseThdjP > . Meanwhile noiseThd verify the characters

whose position has been determined, and judge whether the width meets . If it is satisfied, the result is character, else it is noise. It needs to segment again.

W cWidth

Using the method of projection and standard vehicle plate’s prior knowledge to segment character and modify it, the precision is ensured. It can process some cases of vehicle plate images such as adhesion characters, plates of left and right deviation and minor noise interference.

III. EXTRACT CHARACTER FEATURE

The aim of character feature extraction is to find the most efficient feature from lots of features. So how to choose an efficient feature from many features become the key to recognize character. Usually the method of coarse grid feature extracting is defined that the characters to recognize should be normalized first by size and position, and divided into N Ngrids equally, then calculate the number of white pixels in grids in turn to get one N N dimension grid feature by numeric form. But coarse grid feature belongs to local gray feature, it can show total shape distribution of character, but the ability of anti-disturbance of stroke position change is poor and decreases the recognition rate. What’s more, coarse grids features ignore some detail features which are very important to character classification of similar structure.

Because the vehicle plate character recognition belongs to minimal classification and this kind of image is disturbed by environmental noise inevitably, every pixel of normalized character dot matrix is applied as one grid, that is to say, extract character original feature and input them into neural network classifier to classify characters. This method scan images row by row and column by column in the order of left to right and up to down, then calculate pixel gray. The statistic result is regard as feature vector. The vector’s dimension is defined as width multiplying length. In the article first normalize the vehicle plate character to binary images by size of 24 15, extract 360 feature points by the coarse grid method and input them into neural network classifier to sort.

87

IV. NEURAL NETWORK RECOGNITION ALGORITHM

A. Characteristics of neural network algorithm Neural network is the hot research domain and its

application is extensive. There are many kinds of neural network models. The radial basis function neural network (RBF) is a typical local approximation neural network and it is better than back propagation (BP) neural network in such abilities as approximation, classification and study [8]. The design uses RBF as plate character classifier. RBF is a feed-forward neural network and has simple structure. It also has other self features such as optimal approximation and no local minimum, faster convergent speed and simpler topology structure.

B. Design of RBF neural network 1) Experiment samples choice

The experiment samples are color vehicle images (including static and dynamic vehicle). Firstly, the collected images are preprocessed such as graying, binary, inclination emendation and getting rid of interference. Apply the vertical projection method and self characteristics to segment single character, next extract features to normalize them into character matrix of 24 15. Then make them as neural network input to train RBF neural network. Last use the trained RBF neural network to recognize character.

The training samples are selected from the segmented characters, every character choose 5 different samples and put their corresponding vectors into RBF neural network to train, the left characters are regarded as test samples.

2) Determine the reference of RBF neural network a) Determine the number of input nodes

The neutron number between input layer and output layer is decided by problem itself, and the dimension of coarse grid feature which is selected from the characters to be recognized give the neutron number of input layer. In this vehicle plate recognition system the character is normalized by size of 24 15, every pixel is one grid, and the neutron number of input layer is 360.

b) Determine the number of hidden and output nodes In k-means clustering method the initial hidden node

number is equal to output node number. To the modified study method the hidden node number is always equal to output node number. The typical method to design neural network model classifier is to train its model samples and class label, then use the mode of “1 selected from M” to represent goal vector. So the neutron number of output layer is M which is the class number to be recognized, every neutron stands for a goal class. During this design 1 represents target class and 0 stands for non-target class if using binary to train network. The mode of “1 selected from M” can be implemented to describe target vector in number network, letter network, number and letter network.

c) Determine the hidden function center cj and radius

Determine the hidden function center by basis of jc k-means clustering method and modified study method, then calculate j by the (5):

kcx

k

kjKj

11

2

)(

d) Train the connection weight value from hidden layer to output layer

Due to all the weight value is 1 from hidden to output layer, every hidden node output can be acquired after giving input vector and every node function of hidden layer. Then train RBF neural network using the linear least square method to obtain the connection weight value. So RBF neural network model is established and used as classifier of vehicle plate character input later.

e) Recognize test samples After determining the neural network model, the test

samples are put into network to test. When one character vector is input, the sort number is obtained after calculating in hidden and output layer. Compare this sort number to the self sort one of input feature vectors, you can get the recognition character.

V. SIMULATION RESULTS AND ANALYSIS

In this article the samples are some color vehicle plate images which have different size, different clarity and inclination. Using the former steps introduced to fulfill plate location, the result is shown in Fig.2 (only select one result from others). Using the neural network function change the located image into gray image (shown in Fig.3) and binary image (shown in Fig.4 .

Figure 2. Located image

Figure 3. Gray image

Figure 4. Binary image

Preprocess the binary image, the result is shown in Fig.5. Then apply the vertical projection method introduced in section 2 to segment character, the result can come up to the practical standard and solve usual problems. Take the vehicle plate “LuB EB753” as example, the segmentation result is shown in Fig.6.

88

The characters are needed to preprocess before feature extraction. Normalize the character’s size and position, and character dot matrix is 24 15. Then extract the original feature which is showed in Fig.7 (only show three characters).

The features of number 0-9 and letter A-Z are extracted from the segmented images and normalize them, then make them as input of RBF neural network to train. Note that the results show input signal P is a matrix of 360 36, target vector T is a unit diagonal matrix of 36 36. At last the trained RBF neural network is used to recognize character, the results show that the feature vector of character to recognize are the corresponding column vectors in target vector T.

Figure 5. Images after preprocessing

Figure 6. Characters after segmentation

Figure 7. Feature extraction diagram

When the character to recognize is B, the output vector is I0=[0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] T

When the character to recognize is E, the output vector is I1=[0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] T

When the character to recognize is 7, the output vector is I2=[0.00 0.00 0.00 0.00 0.00 0.00 0.00 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] T

When the character to recognize is 5, the output vector is I3=[0.00 0.00 0.00 0.00 0.00 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] T

When the character to recognize is 3, the output vector is I4=[0.00 0.00 0.00 1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] T

VI. CONCLUSIONS

Based on the study of plate location, the method of vertical projection information with prior knowledge is proposed to segment character and extract the statistic feature, then use the RBF neural network to recognize with feature vectors as input. The test results show that this method can recognize character correctly, but it has no advantage on processing Chinese character and has to consider more interference such as fouling, occlusion and so on. These cases can be taken into consideration to improve the recognition ability effectively in future work.

VII. REFERENCE

[1] COMELL I P. Optical recognition of motor vehicle license plates [ J ] . IEEE Transactions on Vehicular Technology, 1995, 44 (4), pp. 790-799.

[2] NAITO T. Robust license plate recognition method for passing vehicles under outside environment[J], IEEE Transactions on Vehicular Technology, 2000, 49 (6), pp. 2309-2319.

[3] Wei Huang, Xiaobo Lu. etc, Vehicle plate location based on wavelet and texture analysis[J], Engineering Sciences, 2004, 6(3), pp. 16-22 (in Chinese)

[4] Bo Xing, Dequn Liang, Wenju Li. A novel recognition method of plate number and letter character [J], Journal of Liaoning Normal University (natural science edition), 28(1), 2005, pp. 56-58 (in Chinses)

[5] Shan Gao, Wanchun Liu, Yuwen Zhu. The vehicle plate character segmentation and recognition based on SVM [J], Microelectronics and Computer, 2005, 22(6), pp. 34-36 (in Chinese)

[6] Min Wang, Xinhan Huang, Wu Wei, etc, A vehicle plate character recognition method of pattern matching and neural network [J], Journal of Huazhong University of Science and Technology, 2001, 29(3), pp. 48-50 (in Chinese)

[7] Jie Y, Xu L, Wei G, A recognition of vehicle plate with mathematical morphology and neural networks [J] . Journal of Wuhan University of Technology(Transportation Science & Engineering), 2001, 25 (3), pp. 371-374 (in Chinese)

[8] Chun Wang, Bo Liu, Xinzhi Zhou, Research on vehicle plate character recognition based on BP neural network [J], China Measurement Technology, 2005, 31(1), pp. 26-28 (in Chinese)

89