8
Biosystems Engineering (2005) 92 (2), 135–142 doi:10.1016/j.biosystemseng.2005.06.010 AE—Automation and Emerging Technologies Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm Gong Zhang 1 ; Digvir S. Jayas 2 ; Noel D.G. White 3 1 Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB, Canada R3 T 5V6 2 Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB, Canada R3 T 5V6; e-mail of corresponding author: [email protected] 3 Agriculture & Agri-Food Canada, Winnipeg, MB, Canada R3 T 2M9 (Received 21 October 2004; accepted in revised form 13 June 2005; Published online 15 August 2005) An ellipse-fitting algorithm was developed to separate touching grain kernels in images. The algorithm randomly tracks the edge of touching kernels to find the sample points for fitted ellipses. The fitted ellipses were generated by a direct least-squares ellipse-fitting method. Then, clustering was used to identify the best representative ellipse for each kernel of the touching instance. With representative ellipses, touching grain kernel images were separated by morphology transform. Typical touching kernel patterns of four grain types, namely barley, Canada Western Amber Durum (CWAD) wheat, Canada Western Red Spring (CWRS) wheat, and oats obtained from composite samples from several growing locations across the western Canadian prairies were used to test this algorithm. The accuracies of separation were: 924% (barley), 961% (CWAD), 948% (oats), and 973% (CWRS wheat). The kernels used for the touching grains images were separated physically to create non-touching instances and another set of images of kernels were acquired. Morphological features were extracted from images of physically separated kernels and compared with features extracted from software-separated kernels. To decide if the difference between two kinds of features was significant, the large sample Z test of hypotheses was employed. Except for Fourier descriptor 1 of barley and CWAD, software separation did not change the values of morphological features more than the tolerance limits of the measurement system. To assess the classification capability after software separation, the morphological features extracted from physically separated kernels were used as training and basic testing data sets, and the features from software-separated kernels were used as production testing data sets. A back-propagation neural network was employed for grain type classification with morphological features as inputs. Compared to 971% physically separated grain kernels being correctly classified, the mean classification accuracy for all the software-separated grain types was 966%. The morphological features of software-separated kernels were not distorted during software separation and can be successfully used in grain type classification. r 2005 Silsoe Research Institute. All rights reserved Published by Elsevier Ltd 1. Introduction During grain-handling, information on grain type and grain quality is required at several stages to determine the next stage of handling operations. Currently, the process of manually analysing samples is subjective and is influenced by human factors and working conditions. If a machine vision system could identify the contents of a grain sample quickly and with a high accuracy, it should allow automated systems for grain handling and grain quality monitoring. For example, a machine vision system could be used to collect information on grain types and contamination to automatically decide the type of grain cleaning device and its operating para- meters. This would help increased cleaning throughput and recovery of salvageable grains. Substantial work dealing with the use of grain physical features for classifying grains has been reported in the literature (Luo et al., 1999a, 1999b; Majumdar & Jayas, 2000a, 2000b, 2000c, 2000d; Paliwal et al., 1999; Zayas et al., 1989). One of the constraints of these studies was that grain feature extraction algorithms required all kernels to be non-touching. This was necessary because the clusters of touching kernels made ARTICLE IN PRESS 1537-5110/$30.00 135 r 2005 Silsoe Research Institute. All rights reserved Published by Elsevier Ltd

Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

Embed Size (px)

Citation preview

Page 1: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

doi:10.1016/j.biosystemseng.2005.06.010AE—Automation and Emerging Technologies

Biosystems Engineering (2005) 92 (2), 135–142

Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

Gong Zhang1; Digvir S. Jayas2; Noel D.G. White3

1Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB, Canada R3T 5V62Department of Biosystems Engineering, University of Manitoba, Winnipeg, MB, Canada R3T 5V6;

e-mail of corresponding author: [email protected] & Agri-Food Canada, Winnipeg, MB, Canada R3T 2M9

(Received 21 October 2004; accepted in revised form 13 June 2005; Published online 15 August 2005)

An ellipse-fitting algorithm was developed to separate touching grain kernels in images. The algorithmrandomly tracks the edge of touching kernels to find the sample points for fitted ellipses. The fitted ellipseswere generated by a direct least-squares ellipse-fitting method. Then, clustering was used to identify the bestrepresentative ellipse for each kernel of the touching instance. With representative ellipses, touching grainkernel images were separated by morphology transform. Typical touching kernel patterns of four grain types,namely barley, Canada Western Amber Durum (CWAD) wheat, Canada Western Red Spring (CWRS) wheat,and oats obtained from composite samples from several growing locations across the western Canadianprairies were used to test this algorithm. The accuracies of separation were: 92�4% (barley), 96�1% (CWAD),94�8% (oats), and 97�3% (CWRS wheat). The kernels used for the touching grains images were separatedphysically to create non-touching instances and another set of images of kernels were acquired. Morphologicalfeatures were extracted from images of physically separated kernels and compared with features extractedfrom software-separated kernels. To decide if the difference between two kinds of features was significant, thelarge sample Z test of hypotheses was employed. Except for Fourier descriptor 1 of barley and CWAD,software separation did not change the values of morphological features more than the tolerance limits of themeasurement system. To assess the classification capability after software separation, the morphologicalfeatures extracted from physically separated kernels were used as training and basic testing data sets, and thefeatures from software-separated kernels were used as production testing data sets. A back-propagation neuralnetwork was employed for grain type classification with morphological features as inputs. Compared to97�1% physically separated grain kernels being correctly classified, the mean classification accuracy for all thesoftware-separated grain types was 96�6%. The morphological features of software-separated kernels were notdistorted during software separation and can be successfully used in grain type classification.r 2005 Silsoe Research Institute. All rights reserved

Published by Elsevier Ltd

1. Introduction

During grain-handling, information on grain type andgrain quality is required at several stages to determinethe next stage of handling operations. Currently, theprocess of manually analysing samples is subjective andis influenced by human factors and working conditions.If a machine vision system could identify the contents ofa grain sample quickly and with a high accuracy, itshould allow automated systems for grain handling andgrain quality monitoring. For example, a machine visionsystem could be used to collect information on grain

1537-5110/$30.00 135

types and contamination to automatically decide thetype of grain cleaning device and its operating para-meters. This would help increased cleaning throughputand recovery of salvageable grains.

Substantial work dealing with the use of grainphysical features for classifying grains has been reportedin the literature (Luo et al., 1999a, 1999b; Majumdar &Jayas, 2000a, 2000b, 2000c, 2000d; Paliwal et al., 1999;Zayas et al., 1989). One of the constraints of thesestudies was that grain feature extraction algorithmsrequired all kernels to be non-touching. This wasnecessary because the clusters of touching kernels made

r 2005 Silsoe Research Institute. All rights reserved

Published by Elsevier Ltd

Page 2: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

G. ZHANG ET AL.136

the feature extraction of an individual kernel impossible.For most of these studies, grain kernels were manuallypositioned in a non touching manner for imaging. Inpractice, a grain sample presentation device, such as avibrating bed, may be used to present the grain kernelsin a single-kernel deep layer. However, these devices forpresenting singular kernels still cannot separate alltouching kernels. Crowe et al. (1997) used a samplepresentation system consisting of a vibratory feeder andflat conveyor, to obtain the images of grains. With flow-rates near 60 g/min, about 90% of all kernels werepresented as individual kernels. The majority of touch-ing kernels appeared in groups of two, with less than 4%of all kernels appeared in groups of three or four.Therefore, it is necessary to develop a disconnectingalgorithm to separate touching grain kernel images tosolve this problem. Also, such an algorithm should focuson separating two or three touching kernels, becausemultiple kernels touching instances can be eliminated byusing mechanical systems (Crowe et al., 1997).

A mathematical morphology-based algorithm wasdeveloped and tested for disconnecting the conjointkernel regions in an image of touching grains (Shatadalet al. 1995a, 1995b), but it was time-intensive and didnot perform well on grain kernels with slender shape. Inanother study, grain kernels were approximated asellipsoids and it was assumed every grain kernel couldbe represented by the approximating ellipsoid. Indivi-dual grain kernel identification could be done by findinga fitted ellipse, which ‘represents’ the silhouette of thegrain kernel (Shashidhar et al., 1997). Therefore, theobjectives of this study were to:

(1)

develop and implement an ellipse fitting algorithm toseparate the touching grain kernel regions in imagesand test the effectiveness of the algorithm ondifferent cereal grains, e.g. Canada Western RedSpring (CWRS) wheat, Canada Western AmberDurum (CWAD) wheat, barley, and oats with twoor three touching instances;

(2)

test the integrity of the features after softwareseparation of kernels; and

(3)

determine the discriminating ability of morphologi-cal features of software-separated kernels for theclassification of different cereal grains, e.g. CWRSwheat, CWAD wheat, barley, and oats.

2. Materials and methods

2.1. Imaging system

The hardware of the image acquisition system used inthis study consisted of a colour camera (Model

DXC ¼ 300A, Sony), a camera control unit (ModelCCU-M3, Sony), a diffuse illumination chamber with acircular fluorescent tube (305mm in diameter, 32W,Model FC1279/CW, Philips, Singapore) with light con-troller (Model FX06482/120, Mercron, Richardson, TX),and a colour frame grabber installed in a personalcomputer (PC). The camera captured images from thesamples placed in the illumination chamber. The cameraoutputted three parallel analogue video signals, namely red(R), green (G), and blue (B), corresponding to the threeNational Television System Committee (NTSC) colourprimaries, and a sync signal. The acquired digital imageswere then stored for analysis using an IBM-compatiblepersonal computer. All image-processing algorithms wereprogrammed in Visual C++ (Microsoft).

2.2. Grain samples

The grain samples for this study were obtained fromthe Industry Services Division of the Canadian GrainCommission (Winnipeg, MB). For the 1998 growingyear, clean grain samples of CWRS wheat (Grades 1, 2,and 3), CWAD wheat (Grades 1, 2, and 3), barley(Special Select Malt Barley), and oats (Grade 1) wereused to test the separation algorithm. Samples werecollected from 30 growing regions of western Canada.These growing regions were chosen using the climaticsubdivisions of the Canadian Prairies (Putnam &Putnam, 1970). All the samples were mixed and 2000kernels of each grain type were randomly picked andwere used to create 1000 touching instances for eachgrain type. Around 10% of all touching instances weretriple touching kernels; other instances were doubletouching kernels. For the touching instances, one grainkernel was randomly dropped as the centre, and otherkernels were manually placed to touch the centre kernelto simulate possible different touching instances. Touch-ing instances were created to result in a point or a linecontact or in between possibilities. Care was taken notto let kernels overlap. After the image of the touchingkernels was acquired, the kernels were then manuallyseparated without disturbing the orientation of thekernels. Then, an image of manually separated kernelswas also acquired with the same hardware and softwaresettings.

3. Algorithm

3.1. Segmentation and boundary extraction

An adaptive thresholding technique based on R, G, Bvalues of image pixels and hue histograms was used to

Page 3: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

SEPARATION OF TOUCHING GRAIN KERNELS 137

determine the threshold value (Luo et al., 1997). Afterthe threshold value was determined, the pixels with greyvalue higher than threshold were given the value of oneas objects; others had the value of zero as background(Fig. 1(a)). Therefore, the colour image was transferredto a binary image. If a small dark region within anobject fell below the threshold, this region was assignedas background and represents a ‘hole’. With ‘regiongrowing’ to find all inter-connected background pixels,the ‘holes’ were identified and filled. An object boundaryis the closed edge that surrounds a region. If neighbour-ing pixels have different values (0 or 1), they representedge points. When the object boundary was extracted bythe edge detection operator, the object edge pixels weretracked and stored in an ordered points list. Thisordered points list was used to select sample points forellipse fitting. The result of edge tracking is illustratedin Fig. 1(b).

Fig. 1. (a) The touching grain kernels of barley; (b) imageafter edge tracking; (c) finding fitted ellipses; (d) finding the

representative fitted ellipses

3.2. Ellipse fitting

After thresholding, the touching instance with multi-ple grain kernels was then considered as one object inthe binary image. Since the grain kernels in the imageare similar to an ellipsoidal shape, to separate thetouching kernels, an ellipse fitting algorithm wasimplemented to approximate every kernel.

An ellipse is a conic that can be described by animplicit second-order polynomial:

F ðx; yÞ ¼ ax2 þ bxy þ cy2 þ dx þ ey þ f ¼ 0 (1)

with an ellipse-specific constraint:

b2� 4aco0 (2)

where a, b, c, d, e, f are coefficients of the ellipse and(x, y) are coordinates of sample points lying on it. Thepolynomial F(x, y) is called the algebraic distance of apoint (x, y) to the conic F ðx; yÞ ¼ 0: By introducingvectors

a ¼ ½a; b; c; d; e; f �T

x ¼ ½x2;xy; y2; x; y; 1�(3)

where T is the transpose operator.Equation (1) can be rewritten to the vector as

FaðxÞ ¼ x�a ¼ 0 (4)

Fitting an ellipse to a general conic is to minimise thealgebraic distance over the set of N data points inthe least-square sense. To ensure an ellipse-specificity ofthe solution, because the vector a can be arbitrarilyscaled, the following constraint equation can beconsidered instead of Eqn (2) (Fitzgibbon et al., 1996):

4ac � b2¼ 1 (5)

This constraint could be expressed as aTCa ¼ 1, whereconstraint matrix C is of the size 6 6:

C ¼

0 0 2 0 0 0

0 �1 0 0 0 0

2 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

0 0 0 0 0 0

2666666664

3777777775

(6)

The vector a could be calculated based on followingequations:

Sa ¼ lCa

aTCa ¼ 1(7)

where S is the scatter matrix of the size 6 6; and l is aneigen value for S:

S ¼ DT D (8)

Page 4: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

G. ZHANG ET AL.138

and the design matrix D of the size N 6 is

D ¼

x21 x1y1 y2

1 x1 y1 1

..

. ... ..

. ... ..

. ...

x2i xiyi y2

i xi yi 1

..

. ... ..

. ... ..

. ...

x2N xNyN y2

N xN yN 1

26666666664

37777777775

(9)

The ellipse-specific fitting problem can be solved byminimising constraint, E ¼ ||Da||2 under the constraintEqn (5). Introducing the Lagrange multiplier, it leads tothe system Eqn (7), which is a generalized eigen vectorsystem. The chosen eigenvector ak (k is the eigrenvectornumber) corresponds to the minimal positive eigenvalue. The minimal value of ||Da||2 is then directly givenby Eqn (10) (Fitzgibbon et al., 1996):

Dak k2 ¼ aTDTDa ¼ aTSa ¼ laTCa ¼ l (10)

Then, the solution of the minimisation problemrepresents the best-fit ellipse for the given set of points.Every time, six edge sample points were randomlyselected from the ordered edge points list for one ellipsefitting. The result of finding fitted ellipses is shown inFig. 1(c).

3.3. Determining the representative fitted ellipse for each

kernel

Every trial of the ellipse fitting procedure created anellipse for the touching kernel groups. For a particulargroup of touching kernels, 100 fitted ellipses weregenerated by ellipse fitting. When a set of similarrepresentative ellipses for every individual kernel in thetouching group were generated, some extraneous ellipseswere also generated. When all fitted ellipses werecreated, two selected criteria were applied to eliminateinappropriate fitted ellipses. These criteria were asfollows.

(a)

All fitted ellipses must meet a1oa/boa2, where a isthe short axis of the ellipse, b is the long axis of theellipse. After numerous trials to different types ofkernels, the threshold a1 was determined as 0�3, andthe threshold a2 was determined as 0�9.

(b)

The measurement of overlap, x, between thetouching object and fitted ellipse were limited as

x ¼o \ �

where o is the number of the set of touching objectpixels, and e is the number of the set of pixels of thefitted ellipse. If the value for x40�95; the fittedellipse is a proper fitted ellipse.

In fitted ellipse clustering, if each cluster onlycontained the ellipses similar to one individual grainkernel, the filtered ellipses could be grouped to clustersto determine the kernel numbers in the touchinginstance. For each fitted ellipse, lx, ly, a, b, and y arefive main parameters, where (lx, ly) is the centre of theellipse, a is the short axis of the ellipse, b is the long axisof the ellipse, and y is orientation of the long axis fromthe X axis. With these five parameters, the differenceand similarity of fitted ellipses could be determined.When each parameter of an ellipse was considered as adimension of Euclidean space, that ellipse could beidentified as a point of this Euclidean space R5. Tomeasure the similarity between two fitted ellipses, adistance measure based on the above features space wasapplied and the distance between two ellipses wasdefined as the Euclidean distance d.

The clustering was based on the minimisation of aperformance index, which was defined as the sum ofEuclidean distances from all patterns in a cluster domainto the cluster centre. For two ellipses, xi and xj , if d(xi,xj)od, these two ellipses belonged to one cluster, whered is a threshold determined by Euclidean distancemeasurement of 800 touching instances. When allellipses were classified, K initial clusters were generated.With each cluster, the cluster vector zj was obtained by:

zj ¼ 1=nPn

i¼1

xi where, n is the number of ellipses in each

cluster, j is the cluster number. If a cluster had less thanfive ellipses, this cluster was considered as an inappropri-ate ellipse cluster and excluded. After repeating clusteranalysis, some of the clusters coalesced to create a newsingle cluster containing a new representative ellipse. Inthe end, the sets of clusters were built up and the centrepattern of each cluster was the representative ellipse forthat cluster. When representative ellipse had been assignedto each cluster, every kernel of the touching grain groupwas replaced by a fitted ellipse. Moreover, when theserepresentative ellipses were generated, these fitted ellipseswere not allowed to touch each other (Fig. 1(d)).

3.4. Creating the sequential thickening regions

After every representative ellipse for each kernel of atouching case was determined, a mathematical morpho-logical method was used to dilate the fitted ellipses.During the dilation, these dilated ellipses were notallowed to join each other.

This method was used to grow the ellipse and preventthe neighbouring expanding components from joiningtogether. When the ellipse fitting was finished, the pixelsof every representative ellipse were assigned differentobject number and the background pixels were given the

Page 5: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

Fig. 3. Image morphological logic with grain kernels

SEPARATION OF TOUCHING GRAIN KERNELS 139

value zero. When the ellipses were dilated, the pixel withan object number could not have a neighbour pixel withdifferent object number. Thus, these fitted ellipses werenot allowed to touch each other. This operation applieda mixed structuring element, L ¼ (l1, l2) (Serra, 1982).This structuring element is:

l2 l2 l2

n l2 n

l1 l1 l1

where: l1 is the hit element and l2 is the miss element, and� is blank element. That is, only those pixels areincluded in the hit-or-miss transform of an image withmixed structuring element, L, where simultaneously thel1 locations hit the foreground of the image and l2locations miss the foreground. For a pixel, if, with eightof its neighbours, at least three neighbours as defined byl1 locations are ‘on’ and at least three other neighboursas defined by l2 locations are ‘off’, this pixel was turned‘on’. For sequential thickening, the above configurationof the structuring element L, and seven other rotationsof this grid were used (Serra, 1982). This logic imposedduring growing the fitted ellipses prevented their merger.Sequential thickening was repeated one hundred timesfor each fitted ellipse to make sure the dilating regionswere big enough to cover the silhouette of the originalkernels. The example of dilation is shown in Fig. 2.

3.5. Separating the touching grain kernel regions

After dilation, the dilated ellipses can cover thesilhouette of the grain kernel of the touching groups(Fig. 3). As all these dilated ellipses are separated, usingthe logic ‘AND’ with dilated ellipses and the originaltouching group, the touching isthmus between thekernels were identified, and the clusters of touchingkernel regions in the image were separated. Theseparation result is shown in Fig. 4.

3.6. Testing the separation algorithm

The physical dimensional measures that characterisethe appearance of an object are called morphological

Fig. 2. Fitted ellipse dilation

features in the image. Majumdar and Jayas (2000a)developed an algorithm based on morphological fea-tures to classify individual kernels of CWRS wheat,CWAD wheat, barley, oats, and rye. Forty-threemorphological features were extracted for the discrimi-nate analysis. When the morphology model with the 10most significant morphological features was tested, theclassification accuracies of CWRS wheat, CWADwheat, barley, oats, and rye were 98�9, 93�7, 96�8, 99�9,and 81�6%, respectively. In their morphological featuresmodel, several of the morphological features are derivedfrom a select number of basic features. Assessment ofchanges to the basic features can give a good ideawhether software separation affects morphologicalfeatures. Therefore, 13 morphological features werepicked to assess separation algorithm: area, perimeter,maximum radius, minimum radius, mean radius, majoraxis, minor axis, spatial moment 1, spatial moment 2,Fourier transcript 1, Fourier transcript 2, major length,minor length.

The separation algorithm was applied to the images oftouching kernels of CWRS wheat, CWAD wheat,barley, and oats. For each grain type, 1000 touching

Fig. 4. (a) The touching grain kernels of barley; (b) kernelsfollowing software separation

Page 6: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

Fig. 5. Improper placement of the separation lines

G. ZHANG ET AL.140

cases were tested and the effectiveness of the separationalgorithm in separating the touching kernels wasdetermined. In general, the unsuccessful separatedtouching grain kernels can be visually identified.However, some small distortions of grain regions aftersoftware separation could be introduced and may affectthe value of extracted morphological features. After thegrain kernels were separated, each separated kernel wasprocessed to extract features for identification. If themorphological features were distorted during separa-tion, the identification of the grain kernel would not beaccurate. Therefore, the difference in morphologicalfeatures between software separated and physicallyseparated kernels were compared to test the effectivenessof the separation algorithm.

For software separation, any case of touching kernels,which were not separated or were distorted by improperplacement of the separation lines, were consideredunsuccessfully separated cases or kernels. For separa-tion algorithm evaluation, one separation error isconsidered as unsuccessful separation for the touchingcase with two or three touching kernels.

To assess the classification capability after softwareseparation, the morphological features extracted fromphysically separated kernels were used as training andbasic testing data sets; the features from software-separated kernels were the production testing data. Afour-layer back-propagation neural network (BPNN)was developed as grain type classifier based onmorphological features. Training and basic testing wasdone on 8000 physically separated kernels (i.e. 2000kernels of each grain type), and the entire process wasrepeated three times using different training and testingdata sets. Each time, 50% randomly selected physicallyseparated kernels were used as training data sets,another 50% physically separated kernels were used asbasic testing data. The network was trained for 1500epochs and was then applied on production data setconsisting of 8000 software-separated kernels (i.e. 2000kernels of each grain type).

4. Result and discussion

4.1. The results of software separation

The success rate in separating all grain touching caseswas 97�3% for CWRS wheat, 92�4% for barley, 96�1%for CWAD, and 94�8% for oats. When the mathema-tical morphological separating algorithm (Shatadal et

al., 1995a, 1995b) was applied to the touching cases oflong ellipsoid kernels with a longer isthmus area, such asoats, it often failed because of over erosion. With theellipse-fitting algorithm, this problem was solved by

using a fitted ellipse to isolate the isthmus area.Therefore, the ellipse-fitting algorithm performed betterin separating the touching oat kernels with 94�8%accuracy, compared to 79% separation accuracy withthe mathematical morphological based separation algo-rithm of Shatadal et al. (1995a).

The ellipse-fitting algorithm also had some limita-tions. During the separation process, if the representa-tive ellipses were improperly selected during clustering,the separation line could not be placed correctly.Figure 5 shows an example where the separation linewas placed within a kernel region rather than at theisthmus. This happened because one representativeellipse was too small during dilation. The bigger fittedellipse dilated too much to set the separation line at thekernel region instead of the isthmus. The separationsuccess rate for barley kernels was the lower becausesome barley kernels were not approximated as ellip-soids. For those kernels with irregular shape, someimproperly fitted ellipses were filtered during overlapmeasurement. However, some smaller improperly fittedellipses may have passed the overlap filter and clusteredto the representative ellipse. This would cause theseparation line to move toward the smaller fitted ellipseduring dilation and place the separation line within thekernel region.

4.2. Effectiveness of morphological features of

software-separated kernels

If the separation algorithm worked well, after soft-ware separation, those morphological features shouldnot change considerably from their true values. The truevalue of a morphological feature is a value extracted

Page 7: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

Table 2Classification accuracies of software-separated grain kernels

using neural network for three trials

Cereal Classification accuracy, %

Trial 1 Trial 2 Trial 3 Mean

Barley 96�6 96�9 97�4 97�0CWAD 94�3 94�8 94�9 94�7CWRS 98�2 97�5 97�3 97�7Oats 96�9 97�5 97�2 97�2

Table 1

Classification accuracies of physically separated grain kernels

using neural network for three trials

Cereal Classification accuracy, %

Trial 1 Trial 2 Trial 3 Mean

Barley 97�8 96�6 97�1 97�2CWAD 95�9 95�2 95�4 95�5CWRS 98�1 98�3 97�8 98�1Oats 97�4 97�7 97�1 97�4

CWAD, Canada Western Amber Durum; CWRS, Canada Western

Red Spring.

SEPARATION OF TOUCHING GRAIN KERNELS 141

from the image when the kernels were physicallyseparated.

To demonstrate the effectiveness of the separationalgorithm, successfully separated samples (973 touchinginstances of CWRS wheat, 924 touching instances ofbarley, 961 touching instances of CWAD barley, and948 touching instances of oats) were used. For eachkernel, one set of morphological features after physicalseparation was recorded as an initial set of features, andanother set of morphological features of the same kernelwas extracted after software separation.

The difference in the features between the physicallyseparated and software-separated kernels were used totest the effectiveness of the ellipse-fitting algorithm. Todecide if the difference between two kinds of featureswas significant, the large sample Z test (Montgomery,2000) of hypotheses was employed.

Except for Fourier transcript 1 of barley and CWAD,there was no difference for all other morphologicalfeatures for all grain types between physically separatedand software-separated kernels (with 95% confidence).Therefore, software separation did not change thevalues of the morphological features of CWRS wheat,CWAD wheat, barley, and oats.

CWAD, Canada Western Amber Durum; CWRS, Canada Western

Red Spring.

4.3. Classification of the software-separated kernels

with an artificial neural network

The summarised results of classification analysis areshown in Tables 1 and 2. Compared to 97�1% physicallyseparated grain kernels being correctly classified, themean classification accuracy for all the software-separated grain types was 96�6%. Because the shapeand size of CWAD kernels varied a lot, the classificationaccuracy for CWAD was the lowest among foursoftware-separated grain types. One reason for addi-tional misclassification caused by software separationwas the use of physically separated kernels as a trainingand basic testing sets and the software-separated kernelsas a production test set. Another reason for thisadditional misclassification was CWAD misclassifiedas CWRS wheat due to improper separation. Resultspresented in Tables 1 and 2 show that morphologicalfeatures of software-separated kernels were not distortedduring software separation and can be successfully usedin grain type classification.

If a commercial processor is designed to use with thisalgorithm, the processing speed can be accelerated tomeet the real-time processing requirement. Finding therepresentative fitted ellipse with clustering is essential inthis algorithm, therefore more clustering techniquesneed to be tested to identify the best clusteringtechnique.

5. Conclusion

Touching grain kernels separation is essential forautomated grain sample analysis. The ellipse-fittingalgorithm was developed for separating the touchinggrain kernels in an image. A back-propagation neuralnetwork was also developed for grain type classificationbased on the morphological features model. The studyshows that software separation did not distort morpho-logical features of kernels and can be successfully usedfor grain classification. Following conclusions can bedrawn from this research.

(1)

The ellipse-fitting algorithm successfully separatedtouching grain instances for Canada Western RedSpring (CWRS) wheat, barley, Canada WesternAmber Durum (CWAD) wheat, and oats with97�3%, 92�4%, 96�1%, and 94�8%, respectively.

(2)

Except for Fourier descriptor 1 of barley andCWAD wheat, software separation with the ellipse-fitting algorithm did not change the value ofmorphological features of CWRS wheat, CWADwheat, barley, and oats within the tolerance limits ofthe measurement system.

(3)

The mean classification accuracy for all the soft-ware-separated grain kernels was 96�6%.
Page 8: Separation of Touching Grain Kernels in an Image by Ellipse Fitting Algorithm

ARTICLE IN PRESS

G. ZHANG ET AL.142

Acknowledgements

We thank the Canada Research Chairs Program forfunding this study.

References

Crowe T G; Luo X; Jayas D S; Bulley N R (1997). Colour line-scan imaging of cereal grain kernels. Applied Engineering inAgriculture, 13(5), 689–694

Fitzgibbon AW; Pilu M; Fischer R B (1996). Direct least squaresfitting of ellipses. Proceedings of the 13th InternationalConference on Pattern Recognition Vienna, pp. 253–257

Luo X Y; Jayas D S; Crowe T G; Bulley N R (1997).Evaluation of light sources for machine vision. CanadianAgricultural Engineering, 39(4), 309–315

Luo X Y; Jayas D S; Symons S J (1999a). Identification ofdamaged kernels in wheat using a colour machine visionsystem. Journal of Cereal Science, 30(1), 49–59

Luo X Y; Jayas D S; Symons S J (1999b). Comparison ofstatistical and neural network methods for classifying cerealgrains using machine vision. Transactions of the ASAE,42(2), 413–419

Majumdar S; Jayas D S (2000a). Classification of cereal grainsusing machine vision. I. Morphology models. Transactionsof the ASAE, 43(6), 1669–1675

Majumdar S; Jayas D S (2000b). Classification of cereal grainsusing machine vision. II. Colour models. Transactions of theASAE, 43(6), 1677–1680

Majumdar S; Jayas D S (2000c). Classification of cereal grainsusing machine vision. III. Texture models. Transactions ofthe ASAE, 43(6), 1681–1687

Majumdar S; Jayas D S (2000d). Classification of cereal grainsusing machine vision. IV. Combined morphology, colour,and texture models. Transactions of the ASAE, 43(6),1689–1694

Montgomery D C (2000). Design and Analysis of Experiments.Wiley, Chichester

Paliwal J; Shashidhar N S; Jayas D S (1999). Grain kernelidentification using kernel signature. Transactions of theASAE, 42(6), 1921–1924

Putnam D F; Putnam R G (1970). Canada: A RegionalAnalysis. Dent and Sons, Inc., Toronto

Serra J (1982). Image Analysis and Mathematic Morphology.Academic Press, New York

Shashidhar N S; Jayas D S; Crowe T G; Bulley N R (1997).Processing of digital images of touching kernels byellipse fitting. Canadian Agricultural Engineering, 39,139–142

Shatadal P; Jayas D S; Bulley N R (1995a). Digital imageanalysis for software separation and classification oftouching grains: I. Disconnect algorithm. Transactions ofthe ASAE, 38(2), 635–643

Shatadal P; Jayas D S; Bulley N R (1995b). Digital imageanalysis for software separation and classification oftouching grains: II. Classification. Transactions of theASAE, 38(2), 645–649

Zayas I; Pomeranz Y; Lai F S (1989). Discrimination of wheatand non- wheat components in grain samples by imageanalysis. Cereal Chemistry, 66, 233–237