Intensiﬁed fuzzy clusters for classifying plant, soil, and ...docsagencia.cnptia.embrapa.br/informatica/sdarticle.pdf · Computers and Electronics in Agriculture 42 (2004) 161–180

Computers and Electronics in Agriculture 42 (2004) 161–180

Intensified fuzzy clusters for classifying plant, soil,and residue regions of interest from color images

George. E. Meyer∗, Joao Camargo Neto, David D. Jones,Timothy W. Hindman

Biological Systems Engineering, University of Nebraska, 250 L.W. Chase Hall,Lincoln, NE 68583-0726, USA

Received 18 December 2002; received in revised form 7 July 2003; accepted 4 August 2003

Abstract

Fuzzy excess red (ExR) and excess green (ExG) indices and clustering algorithms: fuzzy c-means(FCM) and Gustafson–Kessel (GK) were studied for unsupervised classification of hidden and promi-nent regions of interest (ROI) in color images. Images included sunflower, redroot pigweed, soybean,and velvet leaf plants, against bare clay soil, corn residue and wheat residue, typical of the Great Plains.Indices and clusters were enhanced with Zadeh’s (Z) fuzzy intensification technique. Enhanced ROIswere sorted by degree of fuzziness, and recombined into labeled, false-color class images. ROIs withthe lowest degree of fuzziness were consistently found to be plant clusters with some of the methods.The ZExG index only classified plant ROIs correctly at 76% (newly emerged) and 74% (young plants)for soil backgrounds, 55–65% for corn residue, and only 12% for young plants with wheat straw. TheZExR index failed for almost all categories, except bare soil. The ZFCM clustering algorithm correctlyclassified plants from 10 to 69% in bare soil, but failed for plants in corn and wheat residue. The ZGKalgorithm classified plants from 16 to 96% in bare soil, and corn residue plants as high as 95%, andwheat straw plants as high as 99%, depending on age category and the relative pixel area of plantswithin the image. The ZGK algorithm could be potentially useful for remote sensing, mapping, cropmanagement, weed, and pest control for precision agriculture.© 2003 Elsevier B.V. All rights reserved.

Keywords:Fuzzy clusters; Unsupervised classification; Machine vision; Plants; Soil; Residue

∗ Corresponding author. Tel.:+1-402-472-3377; fax:+1-402-472-6338.E-mail address:[email protected] (G.E. Meyer).

0168-1699/$ – see front matter © 2003 Elsevier B.V. All rights reserved.doi:10.1016/j.compag.2003.08.002

162 G.E. Meyer et al. / Computers and Electronics in Agriculture 42 (2004) 161–180

1. Introduction

Most crops are grown in minimum tillage systems in the upper Midwest of the UnitedStates, which means that crop backgrounds are not bare soil, but are usually covered withvarying levels of weathered plant residue. Some commercial optical plant detection systemshave been claimed as successful weed detection systems, but apparently have only beentested against uniform backgrounds of concrete or bare soil (Hummel and Stoller, 2002).This study reported promising results for soybeans, but not corn. Admittedly there are anumber of factors that could affect the accuracy of a selective green sensor for weed controlin corn and soybeans.Nitsch et al. (1991)showed that plants and wheat straw have verysimilar spectral reflectance patterns in the red and NIR band widths, which can confusesimple plant detection systems. It is doubtful that this early technology will work reliablyfor complex residue backgrounds.

Weed, crop, soil, and residue detection research for site-specific crop management hasrecently focused on remote sensing and machine vision sensing methods (Meyer et al.,1998a,b). Machine vision offers a great potential to extract and identify target features,based on color, shape, and texture (Tillett et al., 2001). However, color feature identificationoften requires a supervised decision approach, usually left to the human operator.Onyangoand Marchant (2001)described color objects as seen by an imaging device as an integrationof the illuminated spectrum, the reflectance spectrum of the scene objects, and spectralproperties of the camera lens, shutter and iris settings. Other imaging systems have proposedusing near infra-red (NIR) bands with color (Onyango and Marchant, 2001). NIR systemsprovide less human intuition, and are more difficult to assess during system integration.NIR imaging systems are not considered further in this paper.

Color is a human visual perception concept (Murch, 1984). The retina of the humaneye contains 4% blue, 32% green, and 64% red cones, although there is a non-uniformspatial cone and rod distribution. The human vision system also integrates tonal variation,probably making color really inseparable.Foveation of the eye or concentration of the mindon specific scene details probably makes visual information content more manageable forhumans (Perry and Geisler, 2002). Red (R), green (G), and blue (B) are useful primary colorswith associated intensity image subsets. Primary colors can elicit the human sensation ofcolor, even with an electronic display device.

Color images yielding R, G, and B intensity subimages can be mathematically trans-formed to hue (H), saturation (S), and intensity (I) subimages (Russ, 1992). Woebbeckeet al. (1995)studied excess green (ExG), computed as (2*G)-R-B and modified hue derivedintensity images. ExG images were found useful for identifying plant regions of inter-est (ROI) with bare soil and some residue backgrounds at about 85% accuracy. However,bright soil or residue pixels which contained high green content (although not appearinggreen to the human eye) tended to provide false plant information.Woebbecke et al. (1995)studied modified hue images as a ROI tool, but found them less successful in discriminat-ing plants from soil and residue backgrounds than ExG. These intensity images tend todiscard background features in the form of near-binary images. Once an image ROI is iso-lated, tonal feature information can be quantified using classical texture analysis (Haralick,1979), fast Fourier transforms, or wavelets (Tian et al., 1999). Shearer and Holmes (1990)used HSI intensity subimages and texture to classify plant species from cropped images of

G.E. Meyer et al. / Computers and Electronics in Agriculture 42 (2004) 161–180 163

ornamental nursery stock. To simulate human color perception,Meyer et al. (1998a)pro-posed another intensity image called excess red (ExR), computed as (1.4*R)-G-B. ExRtakes into account relative proportions of rod and cone sensitivities for red and physiologi-cal green.

Fuzzy color image classification systems (fuzzy inference structures) could be developedand tested with a supervised neural network, using RGB as input data.Bezdek (1993)pre-sented the use of a neural network for fuzzy pattern recognition, generating membershipfunctions, performing fuzzy logic operations, and deriving rule sets.Hemming and Rath(2001)proposed a fuzzy weed classifier that yielded correct classification accuracies be-tween 51 and 95%.Hindman and Meyer (2001)showed that a simple fuzzy logic 27-rulesystem could accurately replace the algebraic ExR index for a green leaf. Overall, patternrecognition processes using fuzzy rules are extensive, requiring large image data sets toinsure that they will work correctly. Fuzzy inference methods will be difficult to accept, ifROIs are not first identified and verified by a human subjective means. That step is the mostcostly and time consuming.

Fuzzy clustering offers an alternative method and is unsupervised. Clustering refers tothe partitioning of data into subclasses (Ross, 1995). Clustering methods may be either hard(crisp classified partitions) or soft (fuzzy distinction). Fuzzy c-means (FCM) clustering hasbeen suggested as a method of choice for obtaining segmented object information from colorimages (Moghaddamzadeh et al., 1998; Lim and Lee, 1990). Data is assigned to a givencluster based on a selection criteria, e.g. minimum distance of the data point from the clustercenter in the feature space.Babuska (1998)discussed six clustering techniques and theirlimitations.Beichel et al. (1999)discussed the use of an unsupervised Gath-Geva cluster-ing method for Landsat thematic mapper (TM) images. Their preliminary data showed thatclassification accuracy reached a maximum value of 86% with only five clusters.Duda et al.(1999)concluded that agglomerative hierarchical and fuzzy-maximum likelihood cluster-ing were superior to c-means and fuzzy c-means clustering, but without much supportingdata.

Tizhoosh (1998)presented procedures for fuzzy clustering, rule-based systems, fuzzygeometry, reduction of fuzziness, measures of fuzziness, and fuzzy measure theory forintensity-based images. Fuzzy image enhancement procedures are a result of redefinition ofclassical image analysis using intensity or tonal pixel values as fuzzy numbers. A derivedfuzzy set of intensity (grayscale) images may be considered as a fuzzy set of intensitysingletons (Ross, 1995). Since color images may be presented as three intensity images, thepremise is to find clusters with similar RGB patterns. Each cluster is defined by a uniqueintensity associated with the cluster center. Membership values are assigned according tothe distance of the pixel RGB value from a cluster center. Similarity or membership isdefined mathematically in terms of distance norms. Zadeh’s intensification operator wasdemonstrated for enhancing simple fuzzy image patterns (Ross, 1995). De and Chatterji(1998)also proposed a fuzzy intensification method. Fuzzy intensification and subsequentevaluation of the degree of fuzziness may be useful tools in ranking clusters for potentialregions of interest in an unsupervised manner.

The objective of this study was to study unsupervised color indices and fuzzy clusteringmethods and determine their accuracy for classifying plant, soil, and residue regions ofinterest.


Table 1Total number of images associated with each background and plant type

Plant common name Background

Bare clay soil Weathered corn residue Fresh wheat residue

Sunflower 35 34 34Pig weed 59 55 56Soybean 68 67 68Velvet leaf 73 66 66

2. Materials and methods

An analysis and comparison of two unsupervised plant indices and two fuzzy clusteringtechniques were performed using a total of 681 digital images, arranged in four replica-tions of multiple class color targets, shown inTable 1. Targets included potted individualplants of sunflower (Helianthus pumilus), red root pigweed (Amarathus retroflexus), soy-bean (Glycine max(L.) merrill), and velvet leaf (Abutilon theophrasti Medicus) againstbackgrounds of bare clay soil, weathered corn stalks, and fresh wheat straw. Plants weregrown from seed to 3 weeks in a greenhouse. Images were acquired across three categoriesof approximate plant developmental stages: stage 1, cotyledons; stage 2, cotyledons and afew real leaves; stage 3, real leaves and petioles (young plant at 3 weeks), shown inTable 2.While the selected categories of stages of plant development were somewhat arbitrary, thenumber of plant pixels (plant size) generally increased from stages 1 to 3.

Images were obtained using a Kodak Digital Science DC120 digital camera (resolutionof 1,228,800 pixels) under direct sunlight at solar noon and a color temperature of about5500 K. The DC120 is a fairly inexpensive, but highly useful camera. For best picture andRGB separation, the DC120 camera was operated in the automatic mode. With automaticoperation, the camera sets its own picture acquisition parameters, including exposure time,F-stop (iris), white balance, and default multi-spot focus (three different locations). TheF-stop is always adjusted automatically by this camera, in either automatic exposure ormanual shutter speed modes, according to the position of the zoom lens (wide angle totelephoto). Digital picture resolution was set to best quality (1280× 960 pixels). Images

Table 2Estimated average amount and range of plant covering each frame (percent pixels)a

Target-plant name Stage

1 2 3

Sunflower 0.12–1.5 1.1–4.2 2.2–15.1Pigweed 0.1–1.0 0.16–7.2 5.3–30.1Soybean 0.3–2.3 0.3–14.5 2.3–55.1Velvet leaf 0.1–??b 1.1–??b 2.5–73.7

a Based on pixels of plant and soil regions using ZExG binary images andREGIONPROPS.b Could not be readily determined from ZExG.


were downloaded using Picture Works PhotoEnhancer®, and the Kodak DC120 TWAINinterface.

Image processing was performed using user-written MATLAB®, Version 6.1 script withfunctions from the Image Processing and Fuzzy Logic Toolboxes (The MathWorks, Inc.,2000). Image preprocessing included importing images and resizing of images. Images wereresized to 320× 200 pixels (≈64,000 pixels) from their original megabyte size to improveprocessing speed. Initial tests indicated that no histogram equalization was needed beforeclustering. Image preprocessing, cluster analysis, classification, and final evaluation wereperformed on a Dell 1.7 GHz Pentium IV Windows 2000 computer. Processing softwarewas written withUICONTROLWindows graphics user interface to assist operation.

Referring toFig. 1, processing always began by extracting RGB triplet values from eachreduced size image. The RGB triplets were first used to compute excess red (ExR) and excessgreen (ExG) intensity images (ExR and ExG were displayed as grayscale images). ExR andExG calculations can result in unusable negative pixel values, each image histogram wasstretched back to basic grayscale domain limits (0–255). ExG was later used to estimate theproportion of plant area within the image frames, using theREGIONPROPSfunction fromMATLAB. Next, the fuzzification of the R, G, B or ExG and ExR intensity images was

Fig. 1. Excess red (ExR), excess green (ExG), fuzzy c-means (FCM), and Gustafson–Kessel (GK) clustering,fuzzification, and classification procedures to determine plant, soil, and residue regions of interest (ROI).


computed as relative intensity singletons and then gathering them into a single membershipset X as a union, given as

X =r−1⋃i=1

c−1⋃j=1

µi,j (1)

whereµi,j = gi,j/gmax is the membership value of the intensity or grayscale value inthe image;gi,j the R, G, B or ExG and ExR intensity pixel value, 0≤ gi,j ≤ gmax, andgmax = 255;r the number of rows andc is the number of columns.

Two fuzzy clustering methods were chosen and compared using only RGB triplet valuesas input (RGB values can be described as a triplet, a data vector, data row, or a variety ofother names). The first clustering method was fuzzy c-means (FCM), according toBezdek(1993). FCM is an iteration cluster process shown byEqs. (2)–(4)for each intensity set ofR, G, and B. The FCM function was provided with the MATLAB Fuzzy Toolbox, and isan unsupervised clustering method. The FCM clustering method assumes the use of equalsized and spherical shaped clusters derived from the data space. Each cluster center is givenas

vli =

∑Nk=1(µ

l−1ik )mzk∑N

k=1(µl−1ik )m

; for 1 ≤ i ≤ c (2)

wherec is the total number of clusters (i is the current cluster);N the number of observations,based on the number of pixels in the image (64,000);m the weighting factor= 2; l theiteration numberl = 1, 2, . . . (not an exponent);zk the data triplet values (R,G, and B) foran input color pixel setZ (0 ≤ k ≤ 64, 000);vi = RGB cluster center values for clusteriandµi,k is the degree of membership of an RGB triplet in clusteri.

FCM can be described as an optimization problem such that the distance between thecluster centers is maximized, while simultaneously minimizing the distance between eachpoint and a cluster center.

The FCM distance normD, representing the distance of each triplet value from therespective cluster center is given as

D2ik = (zk − vl

i)TA(zk − vl

i); for 1 ≤ i ≤ c, 1 ≤ k ≤ N (3)

whereA is the norm inducing matrix (set to identity in this work) and T is the transposeoperation.

The cluster membership functionµ is a partition matrix, given as

µlik = 1∑c

j=1(Dik/Djk)2/(m−1)(4)

After each membership function (Eq. (4)) was updated, new cluster centers were com-puted usingEq. (2), and so forth. The only initialization required is predetermination of afinal number of clusters.

The second method chosen was the Gustafson–Kessel (GK) algorithm (Gustafson andKessel, 1979). This method used a similar prototype cluster, partition matrix, and iteration


as FCM, but used a different distance norm, based on the cluster covariance matrixFi, givenas

Fi =∑N

k=1(µl−1ik )m(zk − vl

i)(zk − vli)

T∑Nk=1(µ

l−1ik )m

; for 1 ≤ i ≤ c (5)

wherec is the total number of clusters (i is the current cluster);N the number of observations,based on the number of pixels in the image (64,000);m the weighting factor= 2; l theiteration numberl = 1, 2, . . . (not an exponent);zk the data triplet values (R, G, and B) foran input color pixel setZ (0 ≤ k ≤ 64,000);vi the RGB cluster center values for clusteriandµi,k is the degree of membership of an RGB triplet in clusteri.

The GK distance normD is given as

D2ik = (zk − vl

i)T[ρi det(Fi)

1/nF−1i ](zk − vl

i); for 1 ≤ i ≤ c, 1 ≤ k ≤ N (6)

where det is the determinant operation on the covariance matrix.The GK method assumes equal cluster volumes of hyper ellipsoidal shape, but shapes are

optimized, perhaps giving it more flexibility than FCM in selecting membership values. Inboth cases, the number of potential clusters must be predetermined. Each clustering algo-rithm required a modified input image data array, using the MATLABRESHAPEfunction.Rows of the reshaped array represented each input pixel as an RGB triplet.

Each clustering method resulted in several subsets of membership values representingcandidate ROIs, with corresponding RGB triplets. Membership values captured in theµ

array, had 64,000 rows and a single column for each cluster. Membership valuesµi rangedfrom 0 (no membership) to 1 (complete membership) in each cluster. Cluster centers werecomputed and represented unique RGB values about which other pixels reside in the cluster,based on membership. Each fuzzy cluster has only one ‘cluster center’, recorded in thev

array. Only RGB triplets were used to determine the structure and spatial location of potentialclusters. No (XYrow–column) values were used from the original image, which emphasizesthat only color, not geometric information was used to classify the images. These clusteringmethods implicitly determine these locations.

Membership values provided fuzzy information about individual pixels for each clusteredROI for a given color image. Each cluster will have a range of membership values, whichcan be defuzzified into separate grayscale intensity images (cluster images). The linearmeasure or degree of fuzzinessγ was given as

γ =[

2

(r − 1)(c − 1)

]r−1∑i=0

c−1∑j=0

min(µi,j, 1 − µi,j) (7)

Next, a new method was developed to identify and rank clusters, based on their degreeof fuzziness. Initialγ values were usually found to be from 0.4 to 0.6 (very fuzzy) foreach cluster. Zadeh’s image intensification was used to modify theµi,j (rather fuzzy), thusrefining potential ROIs. Zadeh’s intensification operator was applied six times to each ExR,ExG, and cluster intensity image, and is given as

µi,j =∣∣∣∣∣ 2[µi,j]2; if 0 ≤ µi,j ≤ 0.5

1 − 2[1 − µi,j]2; if 0.5 ≤ µi,j ≤ 1.0(8)


whereµi,j is a membership value associated with the tonal value of each cluster intensityimage.

Zadeh’s operator adjusts membership values such that lowµi,j values are lowered fur-ther (tending to eliminate these pixels from cluster membership), while highµi,j valueswere increased (reinforcing these pixels as more complete members of the ROI). DuringZadeh’s process,γ is also lowered and can be gagged. As intensification is performedon each index or cluster membership, so that features are mapped into two groups: oneof the high membership region of interest and the other to low membership (those pix-els or triplets that do not belong). In the latter case, it was not immediately apparentwhere these pixels might belong. Therefore, the other regions of interest must be inten-sified to see if these pixels belong to them. A few pixels might remain, not belongingto any region of interest. Repeated intensification resulted in finalγ of less than 0.01,and a set of near-binary clusters. A residual of the intensification process was aγ ar-ray, providing the degree of incremental fuzziness reduction for each cluster or region ofinterest.

Intensified cluster sets cannot be reshaped back into a meaningful RGB image, if thenumber of clusters is greater than three. Therefore, the binarized (intensified) clusters wereeach labeled with unique false color based on their finalγ value. The false-color rankingand labeling were rendered by assigning RED to the lowest finalγ, followed by GREEN,BLUE, YELLOW, CYAN, MAGENTA, WHITE, and finally BLACK (membership un-known) in that order, as shown inTable 3. The labeled clusters were then recombinedinto a new ‘false-color composite image’ for visual assessment. The resultant compositeimage made plant ROIs more or less visually distinct and easily recognizable. Further-more, the false-colored clusters represented a consistent ranking of the ROIs, based ondegree of fuzziness. Finally, inspection, evaluation, and tabulation of the plant ROI classi-fication (as RED) was done by visual comparison of the original color with the clusteredfalse-color images. A value of one was assigned, if the plant was RED or RED–GREEN.A value of zero indicated that the plant was not identified. Average correct classifica-tion rates (CCR) were calculated across the replications using SAS Analyst (SAS Insti-tute, Cary, NC) and PROC FREQ. An analysis of variance was performed with PROCGLM.

Table 3False-color assignments to fuzzy cluster images after Zadeh’s intensification

False color assigned Pixel values Degree of fuzziness,γ

Red Green Blue

RED 255 0 0 LowGREEN 0 255 0BLUE 0 0 255YELLOW 255 255 0 MediumCYAN 0 255 255MAGENTA 255 0 255WHITE 255 255 255 HighBLACK 0 0 0 Unknown


Table 4Analysis of variance for the segmentation and cluster study

Source DF Type III Sum of squares F P > F

Age 1 12.83 12.83 154.43 <0.0001BackG 2 1.48 0.74 8.91 0.0001Cmethod 5 4.66 0.93 11.21 <0.0001Plant 3 1.21 0.40 4.87 0.0022REP 1 0.02 0.02 0.26 0.6108Age× BackG 2 0.29 0.15 1.8 0.1649Age× Cmethod 5 11.79 2.36 28.39 <0.0001Age× Plant 3 1.40 0.47 5.62 0.0008Age× REP 1 0.04 0.04 0.43 0.5112Cmethod× BackG 10 8.92 0.89 10.74 <0.0001Plant× BackG 6 4.47 0.75 8.98 <0.0001REP× BackG 2 0.02 0.01 0.1 0.9013Cmethod× Plant 15 5.93 0.39 4.76 <0.0001REP× Cmethod 5 0.18 0.04 0.42 0.833REP× Plant 3 0.21 0.08 0.86 0.4634Age× Cmethod× BackG 10 15.54 1.55 18.71 <0.0001Age× Plant× BackG 6 4.53 0.75 9.09 <0.0001Age× REP× BackG 2 0.01 0.00 0.03 0.9668Age× Cmethod× Plant 15 6.82 0.45 5.48 <0.0001Age× REP× Cmethod 5 0.17 0.04 0.42 0.8379Age× REP× Plant 3 0.44 0.15 1.76 0.1517Cmethod× Plant× BackG 30 14.94 0.49 6 <0.0001REP× Cmethod× BackG 10 0.65 0.07 0.79 0.6412REP× Plant× BackG 6 0.70 0.11 1.41 0.2068REP× Cmethod× Plant 15 0.70 0.05 0.56 0.9033

Class Levels Values

Cmethod 6 ExG ExR FC3 FC5GK3 GK5

Plant 4 Red pigweed soybeansunflower velvet leaf

BackG 3 Wheat straw cornstalksoil

REPs 4

Total number of observations: 4086; dependent variable: CCR andα = 0.05.

3. Results and discussion

The analysis of variance shown inTable 4indicates the high significance of the clusteringmethod used and the background. Age of plant was significant primarily relative to the sizeof plant in coverage relative to the background. The plant type itself was not as significant,but the interaction of the plant and background was quite significant.

3.1. Excess green and excess red

The RGB indices and two clustering methods provided interesting and enlightening re-sults, toward a possible unsupervised scheme for detection of plants from bare soil, corn


stalks, and wheat straw residue.Fig. 2a and bshows the sequence of the image process-ing events with MATLAB, starting with the original color input, the reduced sized im-age, Zadeh-intensified excess red (ZExR), Zadeh-intensified excess green (ZExR), an RGBpseudo color cluster image, Zadeh-intensified intensity images for each cluster, and the finalimproved false-color cluster image. The degree of fuzziness is shown below each intensi-fied image. ZExR and ZExG were presented only as black and white binary images withonly two areas of interest, i.e., plant (white) and background (black). Intensification andsubsequent defuzzification were deemed successful if the white plant areas, respectively,matched the original green plant areas, and there was minimal “salt and pepper” noise (mild“salt and pepper” noise can be removed with a median filter). Background areas were notalways dark due to brightness of portions of residue showing up as either ZExR or ZExG,respectively. Examples of those binary images meeting or failing the subjective criteria areshown inFig. 2a and b. A clean ZExR or ZExG binary image can be used for shape featureanalysis or as a template for differential texture analysis between the plant and background(Meyer et al., 1998b).

The ZExG index correctly classified (CCR) plants well for later growth stages againstbare soil background with average CCRs from 74.1 to 79.8% (Table 5). Apparent mois-ture or dew on a few soil surfaces tended to obscure the plant, while dry soils performedbetter. The ZExG index classifications were poorer for younger plants. Corn residue pro-vided more challenge with CCRs of only 55.4–65.4%. The ZExG index did not clas-sify well for any of the wheat straw residue backgrounds, with an average CCR of only11.6%. ZExR did not perform well for either corn or wheat straw image sets. With theseresidue backgrounds, there was too much red color which complicated the ZExRdiscrimination.

The ZExG binary images were used to estimate the proportion of plant area in all of theimages over bare soil, as shown inTable 2. Size of a specific ROIs relative to neighboringROIs may be an important factor for successful clustering. Pixel coverage of stage 1 plantsranged from 0.08 to 2%. Stage 2 plant pixel coverage ranged from 0.16 to 8%. Stage3 plant pixel coverage ranged from 2.2 to 55.1%, depending on the type of plant. TheZExG algorithm did not classify Velvet leaf very well, resulting in considerable uncertaintyin this case. An overlap of the percentage of plant pixel coverage for the various stageswould be expected, as categorization of growth stages, especially for weeds is not an exactscience. However, stage 3 plants were visually larger than stages 2 and 1 plants and so forth.Zadeh-intensified ExG identified plants from 75.5% (stage 1) to 74.1% (stage 3) successrates for bare soil backgrounds, 55.4–65.4% for corn residue, but for wheat straw, only stage3 plants at only 11.6%.

3.2. Fuzzy c-means (FCM)

Intensification and subsequent classification were judged successful only if a RED orRED–GREEN plant silhouette occurred and, respectively, matched its counterpart in theoriginal color image. There also had to be minimal “salt and pepper” noise. Zadeh-intensifiedfuzzy c-means (ZFCM) clustering did not classify plants correctly at the stages 1 and 2growth stages, and was only marginally correct for stage 3 plants. Average CCRs across allplant species were either zero or quite variable (bare soil from 9.5 to 68.7%) (Table 6).


Fig. 2. Observation collages of original color image, Zadeh-intensified excess red (ExR) and excess green (ExG)intensity images and three Gustafson–Kessel (GK) fuzzy intensity images of (a) velvet leaf (stage 3) againstsoil back ground and (b) five GK red root pigweed images against wheat straw. Final mapped false-color imagerepresents degrees of final fuzziness (DF): red (plant)< green (soil/residue)< blue (soil/residue), yellow (residue),magenta (residue), cyan (n.a.), white (n.a.), and black (unknown).


Table 5Fuzzy intensified excess green (ZExG) and excess red (ZExR) resultsa

Target-plant name Mean observed correct classification rate (%)b

Plant developmental stagec

1 2 3

ZexR ZexG ZexR ZExG ZExR ZExG

Bare soil backgroundSunflower 2.8 100.0 39.1 95.7 8.7 52.2Pigweed 4.4 80.4 46.8 100.0 83.0 100.0Soybean 37.7 100.0 13.4 91.0 28.0 62.0Velvet leaf 8.4 21.7 25.0 32.5 8.9 82.1

Average 13.3 75.5 31.1 79.8 32.2 74.1Standard deviation 14.2 32.1 12.9 27.5 30.4 18.4

Corn residue backgroundSunflower 0.0 60.6 0.0 100.0 0.0 51.9Pigweed 6.6 54.1 0.0 17.4 0.0 88.0Soybean 0.0 95.4 0.0 98.0 0.0 78.1Velvet leaf 0.0 11.3 0.0 8.8 0.0 43.4


Wheat residue backgroundSunflower 0.0 0.0 0.0 0.0 0.0 0Pigweed 0.0 0.0 0.0 0.0 0.0 14.3Soybean 0.0 0.0 0.0 0.0 0.0 23.3Velvet leaf 0.0 0.0 0.0 0.0 0.0 8.8


a Excess green ((2*G)-R-B) and excess red ((1.4*R)-G-B) images, binarized with Zadeh’s intensificationoperator.

b Rated by observing a collage of the original color and binarized images and SAS PROC FREQ. Salt andpepper noise excluded an image.

c Approximate plant developmental stages: 1, cotyledons; 2, cotyledons and a few real leaves; 3, real leavesand petioles (young plant).

Young and intermediate stage 2 plants were difficult to detect using the ZFCM algo-rithm, resulting in average CCRs of 35% or less. The ZFCM algorithm failed for al-most all plant images with corn or wheat residue backgrounds (the reader is remindedthat the same sets of images were used throughout for ExG, ExR and three and fiveclusters for ZFCM). Furthermore, there were no consistent rankings of degree of clus-ter fuzziness for ZFCM or identification of regions of interest with the residuebackgrounds.

The degree of fuzziness could be quite variable, as shown by the variety of false colorsassigned.Fig. 3 gives a visual comparison of Zadeh-intensified Gustafson–Kessel (ZGK)versus ZFCM false-color images for a stage two sunflower against bare soil. GK is dis-cussed in the next section. It can be seen that the RED intensified area of ZGK is an


Fig. 3. (a) Gustafson–Kessel (GK) and (b) fuzzy c-means (FCM) intensified false-color clusters of potted youngsunflower against a bare soil background. False colors represent degree of final fuzzinessγ: red (plant)< green(shadow)< blue (soil), yellow (soil), magenta (soil), cyan (n.a.), white (n.a.), and black (unknown).


Table 6Fuzzy c-means (ZFCM) clustering resultsa



1 2 3

3d 5d 3d 5d 3d 5d





White residue backgroundSunflower 0.0 0.0 0.0 0.0 18.4 0.0Pigweed 0.0 0.0 0.0 0.0 5.2 0.0Soybean 0.0 0.0 0.0 0.0 2.7 0.0Velvet leaf 0.0 0.0 0.0 0.0 0.0 0.0


a All cluster images intensified with Zadeh’s intensification operator—six times to degree of fuzziness<0.01%.b Rated by observing collages of original color and defuzzified images and SAS PROC FREQ. Cluster of

interest is the plant and must have lowest degree of fuzziness (a plant silhouette is manifested in a false color ofRED or sometimes combination of RED and next highest level of fuzziness—GREEN to be correctly classified).

c Approximate plant developmental stages: 1, cotyledons; 2, cotyledons and a few real leaves; 3, all real leavesand petioles (young plant).

d Number of clusters.

enhanced plant silhouette. The small amount of ZFCM RED of the corresponding clus-ter set is not. Other regions showed various levels of fuzziness, including the pot andsoil surface details. BLACK is the default unassigned ROI, and unexpectedly showedup in some potential plant clusters. Increasing the size of the plant relative to the framedid not improve ZFCM performance. In most cases, the soil surface texture was ren-dered with several false colors, but without consistent patterns. Soil or residue clusterswould require further analysis as to their significance, beyond the scope of this study.Zadeh-intensified FCM algorithm only identified plants with the bare soil backgroundfrom 16.5 to 68.7% with five clusters, and failed for all categories of corn and wheatresidue.


Table 7Intensified fuzzy Gustafson–Kessel (ZGK) clustering resultsa



1 2 3

3d 5d 3d 5d 3d 5d





Wheat residue backgroundSunflower 0.0 0.0 8.7 43.5 97.4 97.4Pigweed 0.0 0.0 8.7 8.7 92.2 100.0Soybean 0.0 0.0 50.0 86.0 79.5 97.3Velvet leaf 0.0 0.0 22.6 32.1 63.2 100.0


a All cluster images intensified with Zadeh’s intensification operator—six times to degree of fuzziness<0.01%.b Rated by observing collages of original color and defuzzified images and SAS PROC FREQ. Cluster of

interest is the plant and must have lowest degree of fuzziness (a plant silhouette is manifested in a false color ofRED or sometimes combination of RED and next highest level of fuzziness—GREEN to be correctly classified).

c Approximate plant developmental stages: 1, cotyledons; 2, cotyledons and a few real leaves; 3, all real leavesand petioles (young plant).

d Number of clusters.

3.3. Gustafson–Kessel (GK)

The ZGK clustering, intensification, and defuzzification algorithm worked very well forstage 3 plants (Table 7). Intensification and subsequent classification were deemed success-ful if a RED or RED–GREEN silhouette (ZGK), respectively, matched the original imageand there was minimal “salt and pepper” noise. Overall, ZGK identified bare soil plantsfrom 46 to 96.3%, corn residue plants from 0 to 94.5%, and wheat straw plants from 0 to98.5%, depending on plant size and age. With the stage 3 plant images, selection of threeor five clusters both classified plants successfully with high percentages. The lowest setswere velvet leaf with 63.1–64.2% CCR. Where the plant existed with a significant propor-


Fig. 4. (a) Gustafson–Kessel (GK) and (b) fuzzy c-means (FCM), intensified false-color clusters of velvet leafagainst a wheat straw background. False colors represent degree of final fuzzinessγ: red (plant)< green (residue)< blue (residue), yellow (residue), magenta (residue), cyan (n.a.), white (n.a.), and black (unknown).


Fig. 5. (a) Gustafson–Kessel (GK) and (b) fuzzy c-means (FCM), intensified false-color clusters of redroot pigweedagainst corn stalk residue background. False colors represent degree of final fuzzinessγ: red (plant)< green(residue)< blue (residue), yellow (residue), magenta (residue), cyan (n.a.), white (n.a.), and black (unknown).


tion of the pixels (>20%) within the image frame, that ROI exhibited the least degree offuzziness and was consistently assigned the false color of RED. A few older or larger plantsmapped both RED and GREEN (the next highest level of fuzziness), resulting primarilyfrom shadows across the leaf. In only a few cases, were plant shadows mapped to GREENon bare soil surfaces. Examples of ZGK images are shown inFigs. 3–5. Fig. 3 shows asunflower, as previously discussed. The ZGK algorithm had acceptable classification ratesfor stage two plants at 87.7 and 80.3% for bare soil and corn residue, respectively. Stage 2plant classifications with wheat residue were lower and variable from 22.5 to 42.6%. Stage1 (small:<2% coverage) plants were difficult to classify in all images, generally with muchlower values than ExG and bare soil.Fig. 4shows a comparison of ZGK and ZFCM imagesfor stage 3 velvet leaf with a wheat straw background. Little or no BLACK (unknown ROIassignment) was found in ZGK images.

Fig. 5a and bshows the comparison of redroot pigweed against corn stalk residue forZFCM and ZGK images. ZFCM did not work at all. ZGK (RED region of interest) classifiedplants well in corn stalk residue with 84.8% (three clusters) and 94.5% (five clusters),respectively. Additional analysis of the corn stalk residue and false-color assignment mightindicate the degree of weathering and other textural details, the significance of which isbeyond the scope of this study. ZGK’s success with stage 3 plants and wheat straw wassignificant. The ZGK algorithm may be therefore an important breakthrough for identifyingplants in wheat straw backgrounds, which has been difficult to accomplish with other opticaland machine vision methods.

4. Conclusions

In this study, unsupervised fuzzy color index and clustering methods were developed andemployed for identifying green plants from soil and residue. These methods did not rely onsaturating the camera field of view or image frame with a single region of interest. Nor, werethe backgrounds just bare soil. Potential ROIs were not predetermined. Zadeh-intensifiedExG ROIs performed reasonably well only for bare soil. Zadeh-intensified ExR failed foralmost all categories, except bare soil. These index methods definitely did not work forwheat straw backgrounds.

The ZFCM algorithm’s poor performance was a complete surprise. However, the successof the ZGK algorithm for unsupervised classification of plants with residue backgroundswas very encouraging. More research should be done here. Apparently, the size of the targetrelative to the background does influence ZGK’s cluster detection performance. In general,when plant pixel coverage was less than 10% in the image, there apparently was not enoughcolor information to cluster them. Regions of green plants were successfully identified andisolated, but not by traditional spectral or color indices, nor by tonal variation (texturalfeatures) of the color planes, nor original grayscale. Additional studies with ZGK or otherclustering methods and number of clusters may indicate whether other targets occupying alarger percentage area of the image frame, would be more likely identified as the cluster ofleast fuzziness.

These results also suggest that certain unsupervised evaluation/discrimination fuzzy clus-tering techniques are possible and useful. There are other unsupervised clustering methods


that should be tested and compared with these results. Fuzzy clustering methods may assistother fuzzy inference or mapping methods for the machine vision detection and discrimina-tion of plants, soil, and residue for remote sensing or real time detection and discrimination.These methods may be useful in assigning meaning to unsupervised clustering and regionsof interest.

Acknowledgements

This article has been approved as Journal Series No. 14731 by the Agricultural ResearchDivision, University of Nebraska, Lincoln. This work was funded in part by USGS SpecialWater Quality and USDA National Needs Fellowship Grants. Mention of specific tradenames is for reference only and not to imply exclusion of others that may be suitable.

References

Babuska, R., 1998. Fuzzy Modeling for Control. Kluwer Academic Publishers, Boston, MA.Beichel, R., Bolter, R., Pinz, A., 1999. Fuzzy clustering of lansat TM scene. In: Proceedings of the IEEE

International Geoscience and Remote Sensing Symposium. IGARSS, Hamburg, Germany, 3 pp.Bezdek, J.C., 1993. A review of probabilistic, fuzzy, and neural models for pattern recognition. Journal of Intelligent

and Fuzzy Systems 1 (1), 1–25.De, T.K., Chatterji, B.N., 1998. An approach to generalized technique for image enhancement using the concept

of fuzzy set. Fuzzy Sets and Systems 25, 145–158.Duda, T., Canty, M., Klaus, D., 1999. Unsupervised land-use classification of multispectral images. A comparison

of conventional and fuzzy-logic based clustering algorithms. In: Proceedings of the IEEE InternationalGeoscience and Remote Sensing Symposium. IGARSS, Hamburg, Germany, 3 pp.

Gustafson, D., Kessel, W., 1979. Fuzzy clustering with a fuzzy covariance matrix. In: Proceedings of the IEEECDC. San Diego, CA, pp. 761–766.

Haralick, R.M., 1979. Statistical and structural approaches to texture. Proceedings of the IEEE 67 (5), 786–804.Hemming, J., Rath, T., 2001. Computer-vision-based weed identification under field conditions using controlled

lighting. Journal of Agricultural Engineering Research 78 (3), 233–243.Hindman, T.W., Meyer, G.E., 2001. Fuzzy logic inference systems for discriminating plants from soil and residue

with machine vision. In: Meyer, G.E., DeShazer, J.A. (Eds.), Precision Agriculture and Biological Quality II.Optical Engineering Press, Bellingham, WA 4203, pp. 111–121 (ISBN: 0-8194-3868-5).

Hummel, J.W., Stoller, E.W., 2002. On-the-Go Weed Sensing and Herbicide Application for the Northern Cornbelt.Pap-Am-Soc-Agric-Eng. St. Joseph, Mich. Society for Engineering in Agriculture, Biological and FoodProcessing and CIGR (021021), 13 pp.

Lim, Y.W., Lee, S.U., 1990. On the color image segmentation algorithm based on the thresholding and the fuzzyc-means techniques. Pattern Recognition 23 (9), 935–952.

Meyer, G.E., Hindman, T., Lakshmi, K., 1998a. Machine vision detection parameters for plant speciesidentification. In: Meyer, G.E., DeShazer, J.A. (Eds.), Precision Agriculture and Biological Quality;Proceedings of the SPIE 3543, 327–335.

Meyer, G.E., Mehta, T., Kocher, M.F., Mortensen, D.A., Samal, A., 1998b. Textural imaging and discriminantanalysis for distinguishing weeds for spot spraying. TRANSACTIONS of the ASAE 41 (4), 1189–1197.

Moghaddamzadeh, A., Goldman, D., Bourbakis, N., 1998. A fuzzy-like approach for smoothing and edge detectionin color images. International Journal of Pattern Recognition and Artificial Intelligence 12 (6), 801–816.

Murch, G.M., 1984. Physiological principles for effective use of color. IEEE Computer Graphics and Applications4 (11), 49–54.


Nitsch, B.B., Von-Bargen, K., Meyer, G.E., Mortensen, D.A., 1991. Visible and Near Infrared Plant, Soil and CropResidue Reflectivity for Weed Sensor Design. Pap-Am-Soc-Agric-Eng. St. Joseph, Mich. American Societyof Agricultural Engineers (913006), 32 pp.

Onyango, C.M., Marchant, J.A., 2001. Physics-based colour image segmentation for scenes containing vegetationand soil. Image and Vision Computing 19, 523–538.

Perry, J.S., Geisler, W.S., 2002. Gaze-contingent real-time simulation of arbitrary visual fields. In: Rogowitz, B.,Pappas, T. (Eds.), SPIE Proceedings on Human Visio Electronic Imaging. San Jose, CA.

Ross, T.J., 1995. Fuzzy Logic with Engineering Applications. McGraw-Hill, New York.Russ, J.C., 1992. The Image Processing Handbook. CRC Press, Baca Roton, FL, p. 35.Shearer, S.A., Holmes, R.G., 1990. Plant identification using color co-occurrence matrices. TRANSACTIONS of

the ASAE 33 (6), 2037–2044.Tian, L., Reid, J.F., Hummel, J.W., 1999. Development of a precision sprayer for site-specific weed management.

TRANSACTIONS of the ASAE 42 (4), 893–900.Tillett, N.D., Hague, T., Miles, S.J., 2001. A field assessment of a potential method for weed and crop mapping

on the basis of crop planting geometry. Computers and Electronics in Agriculture 32 (3), 229–246.Tizhoosh, H.R., 1998. Fuzzy image processing: potentials and state of the art. In: Proceedings of the IIZUKA’98

5th International Conference on Soft Computing, vol. 1. Iizuka, Japan, 16–20 October 1998, pp. 321–324.Woebbecke, D.M., Meyer, G.E., Von Bargen, K., Mortensen, D.A., 1995. Color indices for weed identification

under various soil, residue, and lighting conditions. TRANSACTIONS of the ASAE 38 (1), 259–269.

Documents

Intensiﬁed fuzzy clusters for classifying plant, soil, and ...docsagencia.cnptia.embrapa.br/informatica/sdarticle.pdf · Computers and Electronics in Agriculture 42 (2004) 161–180