A comparison of self-organizing neural networks for fast clustering of radar pulses

Signal Processing 64 (1998) 249—269

A comparison of self-organizing neural networks forfast clustering of radar pulses

Eric Granger!,*, Yvon Savaria!, Pierre Lavoie", Marc-Andre Cantin#

! Electrical and Computer Engineering Department, E! cole Polytechnique de Montre&al, P.O. Box 6079, Station ‘‘Centre-ville’’, Montreal, Qc.,Canada, H3C 3A7

" Defence Research Establishment Ottawa, Department of National Defence, Ottawa, On., Canada, K1A 0Z4# Computer Science Department, Universite& du Que&bec a% Montre&al, P.O. Box 8888, Station A, Montreal, Qc., Canada, H3C 3P8

Received 17 February 1997

Abstract

Four self-organizing neural networks are compared for automatic deinterleaving of radar pulse streams in electronicwarfare systems. The neural networks are the Fuzzy Adaptive Resonance Theory, Fuzzy Min—Max Clustering,Integrated Adaptive Fuzzy Clustering, and Self-Organizing Feature Mapping. Given the need for a clustering procedurethat offers both accurate results and computational efficiency, these four networks are examined from three perspectives— clustering quality, convergence time, and computational complexity. The clustering quality and convergence time aremeasured via computer simulation, using a set of radar pulses collected in the field. Estimation of the worst-case runningtime for each network allows for the assessment of computational complexity. The effect of the pattern presentation orderis analyzed by presenting the data not just in random order, but also in radar-like orders called burst and inter-leaved. ( 1998 Published by Elsevier Science B.V. All rights reserved.

Zusammenfassung

Es werden vier neuronale Netzwerke mit der Eigenschaft der Selbstorganisation verglichen, die dazu dienen,Radarpulsstrome in elektronischen wehrtechnischen Systemen zu entflechten. Die neuronalen Netzwerke reprasentierendie Fuzzy Adaptive Resonance Theory, Fuzzy Min—Max Clustering, Integrated Adaptive Fuzzy Clustering und Self-Organizing Feature Mapping. Wenn ein Bedarf an einer Zuordnungsprozedur besteht, die sowohl hohe Genauigkeit derErgebnisse als auch rechnerische Effizienz bieten soll, dann sind diese vier Netzwerke unter drei Gesichtspunkten zubeurteilen — Zuordnungsqualitat, Konvergenzzeit und rechnerische Komplexitat. Die Zuordnungsqualitat und Konver-genzzeit lassen sich auf dem Wege der Computersimulation messen, indem eine Menge von Radarpulsen gesammeltwerden. Zur Beurteilung der rechnerischen Komplexitat eines jeden Netzwerks wird die erforderliche “Worst-Case”-Rechenzeit geschatzt. Der Einflu{ der Darstellungsweise der Muster wird dadurch analysiert, da{ die Daten nicht nur inzufalliger, sondern auch in radar-ahnlicher Form, namlich als Pakete zeitlich gebundelt und verschachtelt bereitgestelltwerden. ( 1998 Published by Elsevier Science B.V. All rights reserved.

*Corresponding author. E-mail: [email protected].

0165-1684/98/$19.00 ( 1998 Published by Elsevier Science B.V. All rights reserved.PII S 0 1 6 5 - 1 6 8 4 ( 9 7 ) 0 0 1 9 4 - 1

Resume

Quatre reseaux de neurones auto-organisateurs sont compares pour le desentrelacement de sequences d’impulsionsradar dans les systemes de guerre electroniques. Les reseaux neuronaux sont le Fuzzy Adaptive Resonance Theory, leFuzzy Min—Max Clustering, le Integrated Adaptive Fuzzy Clustering et le Self-Organizing Feature Mapping. Etantdonne la necessite de disposer d’une technique de “categorization” qui offre a la fois des resultats precis et une bonneefficacite de calcul, on examine ces quatre reseaux de trois points de vue: la qualite de “categorization”, le temps deconvergence et la complexite algorithmique. Les deux premiers attributs sont mesures par simulation informatique, enutilisant un ensemble d’impulsions radar recueillit sur le chantier. La complexite algorithmique est quant a elle evaluee enestimant le temps de calcul obtenu dans le plus mauvais cas pour chaque reseau. L’influence de l’ordre de presentationdes donnees est analysee en soumettant les donnees de l’ensemble radar non seulement dans un ordre aleatoire mais aussidans des ordres de type radar appeles “burst” et “interleaved”. ( 1998 Published by Elsevier Science B.V. All rightsreserved.

Keywords: Categorization; Clustering; Radar; Deinterleaving; Electronic warfare; Electronic support measures (ESM);Neural networks; Self-organization; Fuzzy Adaptive Resonance Theory; Fuzzy Min—Max Clustering; Integrated Adap-tive Fuzzy Clustering; Self-Organizing Feature Mapping

1. Introduction

The purpose of radar electronic supportmeasures (ESM) is to search for, intercept, locate,and analyze radar signals in the context of militarysurveillance [13,33,39]. When a radar ESM systemis illuminated by several radars, it typically relies onpulse repetition interval (PRI) parameters to dein-terleave the intercepted pulse trains. Unfortunately,the multiplication of intricate PRI patterns inabout every radar model has spurred an inordinatecomplexity in deinterleaving algorithms. Alterna-tive approaches for clustering radar pulses are thusneeded, ones that rely on direction of arrival, fre-quency, pulse width and other such parametersthat can be obtained from individual pulses. Theselection of parameters can vary greatly from oneESM system to the next due to installation, cost,size, tactical and other considerations. Yet, be-yond parameter choice, clustering the interceptedradar pulses constitutes a challenging problem.This clustering must sustain a very high data ratesince, in certain radar bands, the signal densitiesencountered can reach up to 106 radar pulses persecond.

Self-organizing neural networks (SONNs) ap-pear very promising for this type of clusteringapplication, since they can cluster patterns auto-nomously, and lend themselves well to very high-speed, parallel implementation [2,4,24]. Several

innovative SONNs have been reported in the liter-ature, each one demonstrating unique and interest-ing features. This paper presents a comparativestudy of four of them, all potentially suitable forsolving very high throughput clustering problems.They are the Fuzzy Adaptive Resonance Theory[8], Fuzzy Min—Max Clustering [34], IntegratedAdaptive Fuzzy Clustering [21], and Self-Organiz-ing Feature Mapping [23].

The performance of these four neural networks isexamined from three points of view — clusteringquality, convergence time, and computational com-plexity. Indeed, in many practical applications, theaccuracy of clustering results, and the computa-tional efficiency of the clustering procedure areequally important. In this paper, clustering qualityrefers to the degree of similarity between the parti-tions (clusters) produced by a SONN, and a refer-ence partition based on known category labels.This similarity is assessed by applying the RandAdjusted [17] and the Jaccard [10] measures topartitions obtained by computer simulation.Convergence time is defined as the number of suc-cessive presentations of a finite input data setneeded for a SONN’s weight set to stabilize. Thistime is easily determined from computer simula-tion. Computational complexity is estimated fromthe maximum execution time required by a SONNalgorithm to process one input pattern. In order toestimate this worst-case running time, we assume

250 E. Granger et al. / Signal Processing 64 (1998) 249—269

that the algorithm is implemented as a computerprogram running on an idealized random accessmachine (RAM) [9].

The main data set used in our simulationsdescribes electromagnetic pulses transmitted byshipborne navigation radars. These pulses were col-lected from ashore by the Defence Research Estab-lishment Ottawa (DREO) using a directional an-tenna, a superheterodyne tuner, a high-accuracyI/Q demodulator [26], and a two-channel, 10-bitdigital oscilloscope. The radars were observed oneat a time to ensure that each file contained pulsesfrom a single radar. The identity of each ship wasobtained from a harbor control tower and used tolabel each file. These labels can serve as referencesto measure the effectiveness of clustering tech-niques for the application.

In this study, the outcome of numerous com-puter simulations are combined to yield an aver-age similarity measure and convergence time.In addition to the Radar data set described above,two other sets, called Wine Recognition and Iris,are used for comparison. For each set, the patternsare presented in three statistically different orders:random, by burst, or interleaved in a radar-likemanner. The data sets and presentation orderswere selected to illustrate the commonalities anddifferences between the four neural networks.Simulation results and complexity estimates areanalyzed with very high data rate clustering ap-plications in mind.

The rest of this paper is organized as follows. Inthe next section, the main features of the fourSONNs selected for this study are briefly outlined.In Section 3, the methodology used to comparethese SONNs (namely, the performance measures,data sets, and data presentation orders) is de-scribed. Finally, the results are analyzed anddiscussed.

2. Self-organizing neural networks

A clustering method suitable for radar ESMshould have the following properties. First, itshould not require prior knowledge of the numberor characteristics of categories to be formed. Sec-ond, since the variable input arrival rate may reach

106 patterns per second, it should be able to clusternon-stationary streams of input patterns sequen-tially, without requiring their long-term storage.Lastly, the sequence of operations needed for im-plementing the method using current technologyshould lend itself well to high speed hardware real-izations. Given that most of the popular, well-established classical [1,10,11] and fuzzy [5,6] clus-tering algorithms require prior knowledge on eitherthe number or the characteristics of clusters sought,several iterations with the data set, or storage of theentire data set in memory, none of them were con-sidered for this comparison.

The unsupervised learning paradigm used inself-organizing neural networks (SONNs) is relatedto clustering, since it permits the assignment ofadaptively defined categories to unlabeled patterns[16,24]. SONNs appear promising for high datarate sequential clustering applications, since theycan cluster patterns autonomously, in most caseswithout prior knowledge of the number of cate-gories. Moreover, they do not require long-term storage of the input patterns, and permitadaptive determination of the shape, size, number,and placement of categories, while operating inparallel [34]. Self-organizing “neuro-fuzzy” net-works have recently been developed, where fuzzylogic concepts are integrated into the SONNframework. Categories are then modeled asfuzzy sets that allow encoding the input scene’svagueness [27].

Most SONNs are derived from the basic idea ofcompetitive learning [14,15,36], a type of unsuper-vised learning. In short, competitive learning neuralnetworks seek to determine decision regions in theinput pattern space. The most elementary of thesenetworks consists of a single layer of identical out-put neurons, each one fully connected to the inputnodes with feedforward excitory weights, which en-code the categories learned by the network. A cat-egory label is associated with each output neuron,whose features are represented by the numericalvalues of the set of weights connecting it to all inputnodes. When a pattern is presented to the network’sinput nodes, it is propagated through the weights tooutput neurons, which enter a winner-take-all com-petition. The one with the strongest activation (thewinner) is allowed to adapt its set of weights to

E. Granger et al. / Signal Processing 64 (1998) 249—269 251

incorporate the input’s novel characteristics.Through this process, output neurons become se-lectively tuned to respond differently to given inputpatterns, and therefore learn to specialize for re-gions of the input space: they become featuredetectors [32].

Four SONNs that are based on competitivelearning were selected for this study: Fuzzy Adap-tive Resonance Theory (FA) [8], Fuzzy Min—MaxClustering (FMMC) [34], Integrated AdaptiveFuzzy Clustering (IAFC) [21], and Self-OrganizingFeature Mapping (SOFM) [23]. Although the clas-sical ISODATA clustering method [3,10,11,35]does not offer a practical solution to the problem(since it requires significant prior knowledge of thedata), it is included as a reference point for cluster-ing quality comparison, given its widespread use instatistical multivariate data analysis.

2.1. Overview of the four neural networks

The first three SONNs (FA, FMMC and IAFC)subscribe to the basic adaptive resonance theory(ART) [7,15] control structure. Essentially, ARTnetworks categorize familiar inputs by adjustingpreviously learned categories, and create newcategories dynamically in response to inputs dif-ferent enough from those previously seen. Avigilance parameter o regulates the maximumtolerable difference between any two input pat-terns in a same category. Each one of these threeSONNs integrates fuzzy logic concepts into thebinary input ART1 [7] neural network processingframework.

Structurally, the FA neural network, proposedby Carpenter, Grossberg and Rosen [8], consists oftwo layers of neurons that are fully connected: aninput layer F1 with 2M neurons (two per inputfeature) and an output layer F2 with N neurons(one per category). An adaptive weight value w

ji,

represented as a real in the interval [0,1], is asso-ciated with each connection. The indices i andj denote the neurons that belong to the layers F1and F2 respectively. For each neuron j of F2, thecategory prototype vector w

j"(w

j1, w

j2, 2, w

j2M)

contains the set of characteristics defining the cat-egory j. When complement coding is used, this

vector may be interpreted as a hyperrectangle inthe M-dimensional input space.

The FMMC neural network, proposed by Sim-pson [34], consists of two layers of neurons: aninput layer F1 with M neurons (one per inputpattern feature), and an output layer F2 withN neurons (one per category). Two weighted con-nections link every F1 neuron to an F2 neuron.For the F2 output neuron j ( j"1, 2, 2, N), the2 sets of M connected weights encode amin (u

j"(u

j1, u

j2, 2, u

jM)) and a max (�

j"

(vj1

, vj2

, 2, vjM

)) point. These two points definea hyperrectangle fuzzy set B

+, the network’s repres-

entation of category j.FA and FMMC are both fuzzy min—max cluster-

ing networks, whereby category prototype vectorsare represented as fuzzy set hyperrectangles in theM-dimensional input space, each one entirelydefined by a min and a max point [19]. Bothaccomplish fuzzification of the ART1 network byessentially replacing crisp logic AND operatorswith fuzzy logic AND (min) operators. Despitea superficial resemblance to FA, FMMC is a some-what different network, yielding categories asso-ciated to hyperrectangles that never overlap oneanother. A contraction procedure is used to preventthe occurrence of hyperrectangle overlap.

IAFC is a fuzzy clustering algorithm proposedby Kim and Mitra [20,21]. As with FMMC,IAFC contains an M neuron input F1 layer(one per feature), and an N neuron output F2layer (one per category). These neurons are fullyinterconnected by bottom-up and top-downweights. The categories formed, however, are hy-perspherical in shape and are represented as cen-troids in the M-dimensional input space. Twoprototype vectors, �

j"(v

j1, v

j2, 2, v

jM) and b

j"

(bj1

, bj2

, 2, bjM

), are associated with category j’scentroid. The network’s top-down weights v

jien-

code the actual cluster centroids, whereas each bot-tom-up weight b

jiis a normalized mirror of v

ji, for

i"1, 2, 2, M and j"1, 2, 2, N.The objective of IAFC is to obtain improved

nonlinear decision boundaries among closelylocated cluster centroids by introducing a new vigi-lance criterion and a new learning rule into ART1processing. The new vigilance criterion com-bines a fuzzy membership value and the Euclidean


distance to form more flexible categories. The newlearning rule uses a fuzzy membership value, anintracluster membership value, and a function de-pending on the number of data set iterations witha Kohonen-type learning rule.

For our purposes, Kohonen’s Self-OrganizingFeature Mapping (SOFM) [22,23] defines a map-ping from the M-dimensional input space ontoa two-dimensional grid of output neurons, whosesize is user-defined. The SOFM method createsa vector quantizer within the neural structure byadjusting the weights of connections linking theM input neurons of the F1 layer to all the N@ outputF2 map neurons. A prototype vector w

j"

(wj1

, wj2

, 2, wjM

) is associated with every one ofthese map neurons. When an input pattern is pre-sented, each map neuron computes the Euclideandistance of its prototype vector to the input. Theclosest prototype corresponds to the best-matchmap neuron. The weights connected to this neuron,as well as those of the other neurons in its neighbor-hood are adjusted to learn from the input. If thelateral map distance from the best-match neuron tothe location of another one falls within the per-imeter defined by the neighborhood radius, then thisother neuron belongs to the neighborhood.Throughout learning, the neighborhood’s radius,r(t), and the learning rate, a(t), are progressivelydecreased in a linear way from their initial values,r(0) and a(0), until they become equal to 1 and 0,respectively.

As learning evolves, weights become organizedsuch that topographically close map neurons areselectively tuned to inputs that are physically sim-ilar. SOFM clustering is characterized by theformation of a topographic feature map in whichthe spatial coordinates of its neurons correspond tointrinsic features of the input patterns. Therefore,SOFM can be seen as a network producing anunsupervised nonlinear projection of the probabil-ity density function of the high-dimensional inputdata onto a two-dimensional display [23].

2.2. Modifications for radar pulse clustering

Neither one of the four networks in its orig-inal form is suitable for sequential clustering of

radar pulses in practical ESM systems. The aimof this subsection is to propose appropriatemodifications.

FA, FMMC, and IAFC are ART-type networks,whereby output neurons are selected using a poten-tially demanding search process. The computa-tional cost of the iterative category choice andcorresponding expansion testing, through N out-put neurons, is O(N2). However, performing thecategory expansion test on all N output neurons,prior to a direct category choice, can significantlyaccelerate processing in cases where several outputneurons would fail the expansion test. The com-putational cost of the resulting search process isO(N).

The plasticity of ART-type networks like FA andFMMC has been well documented. When an inputresembles none of the stored prototypes, an outputneuron is committed, which amounts to creatinga new category with the input as its prototype. Forradar ESM applications, this implies that thesenetworks can learn a new category from a single, orvery few radar pulses. One drawback is that theycan produce artifact categories, and thus possiblytrigger false alarms, if corrupt pulses are encoun-tered. In practice, some means of managing suchfalse alarms would be required. Notice that IAFC isalso an ART-type network, where plasticity is con-trolled by a learning rate parameter, j, which varieswith the number of data set iterations, and with themismatch between inputs and winning categoryprototypes.

Since the environment around an ESM systemchanges over time as radars come and go, catego-ries that have not been activated for a long timemust be freed. In this paper, the data sets aresufficiently small that we need not bother withcategory removal. In practice, however, categoryreuse would be necessary for maintaining an up-to-date representation of the environment, for ac-curacy, and for making efficient use of computingresources.

SOFM, and to some extent IAFC, permit stablelearning of categories by progressively decreasingplasticity, that is, by reducing the influence of sub-sequent inputs on existing weights. Plasticity iscontrolled by the parameters r(t) and a(t) in SOFM,and j in IAFC. In both cases, the parameter values


are gradually decreased over time, and the net-works eventually loose their ability to react to newinformation. This approach is acceptable for batchdata processing like in this paper. In practice, how-ever, new radars can appear in the theater of opera-tion at any time, and both IAFC and SOFM wouldrequire modification to learn continuously.

In FA, FMMC, and IAFC, input patterns areregrouped into a variable number of categories,each one represented by a prototype vector asso-ciated with a single output neuron. By contrast,SOFM clusters inputs into a fixed number of out-put neurons, organized into a 2-dimensional topo-graphic map. During training, the network adaptsthe prototype vectors for all its map neurons, andassigns an actual data cluster to a region in themap, which may contain one or more map neurons.This process raises the problem of locating catego-ries in the map. Besides, it can be difficult to specifya feature map size when the potential solutionshave a relatively small number (2 or 3) of clusters[29]. A number of interpretation procedures havebeen proposed to assist SOFM for clustering[4,29]. In order to ease the interpretation of thefeature maps in this paper, and to permit a faircomparison with the other ART-type SONNs, anadditional mechanism was incorporated intoSOFMs processing. This mechanism is activatedafter the initial topographical ordering phase hasended, when the map is being fine-tuned for statist-ical accuracy. Define a category center as a mapneuron which has been assigned to more than f in-put patterns. All the other map neurons can berelated to the category centers using a minimumEuclidean distance criterion. Thus, when an inputpattern selects one of these other neurons as thebest-match, the label of the category center neuronwith the closest prototype vector is output by themodified network. This modification makes it pos-sible to locate and count categories, whose numbermay vary as learning unfolds.

3. Comparison method

The methodology used is based on three differentperformance measures — clustering quality, conver-gence time, and computational complexity. This

methodology is different from that of other com-parisons reported in literature (for example,[18,19,30,31,37,38]), in which clustering speedand response time are less critical than in ourapplication. These measures are defined in thissection.

3.1. Clustering quality

A partition of n patterns into K groups definesa clustering. This can be represented as a setA"Ma

1, a

2,2,a

nN, where a

h3[1, K] is the cat-

egory label assigned to pattern h. The degree ofmatch between two clusterings, say A and B, maybe compared by constructing a contingency table,as shown in Fig. 1. In this figure, c

11(c

22) is the

number of pattern pairs that are in a same (differ-ent) cluster in both partitions. The value c

21is the

number of pattern pairs that are placed in a samecluster by A, but in different clusters by B.The value c

12reflects the converse situation. The

sum of all elements m"c11#c

12#c

21#c

22"

n(n!1)/2 is the total number of combinations oftwo out of n patterns. The four variables within thecontingency table have been used to derivemeasures of similarity between two clusteringsA and B [1,10]. These measures are known inpattern recognition literature as external criterionindices, and are used for evaluating the capacity torecover true cluster structure. Based on a previouscomparison of these similarity measures [28], the

Fig. 1. Contingency table used to compare two clusterings.


Rand Adjusted [17], defined by

SRA

(A, B)

"

2(c11

c22!c

12c21

)

2c11

c22#(c

11#c

22)(c

12#c

21)#c2

12#c2

21

(1)

and Jaccard statistic [10], defined by

SJ(A, B)"

c11

c11#c

12#c

21

(2)

have been selected to assess clustering quality forthis study. It is worth noting that variable c

22does

not appear in SJ(A, B).

Since correct classification results are known forthe data sets used, their patterns are all accom-panied by category labels. These labels are withheldfrom the SONN under test, but they provide areference clustering, R, with which a clusteringproduced by computer simulation, A, may becompared. Then, variables c

11and c

22in Fig. 1

represent the number of pattern pairs which areproperly clustered together and apart, respec-tively, while c

12and c

21indicate the improperly

clustered pairs. In this case, Eqs. (1) and (2) yieldscores that describe the quality of the clusteringproduced by a SONN. Both the Rand Adjustedand Jaccard measures yield a score ranging from0 to 1, where 0 denotes maximum dissimilarity, and1 denotes equivalence. The closer a clustering A isto R, the closer the scores are to 1. Notice thedependence of these scores on the number of clus-ters in A and R.

3.2. Convergence time

During a computer simulation, each completepresentation of an input data set to a SONN iscalled an epoch. Convergence time is convenientlymeasured by counting the number of epochsneeded for a SONN to converge. Once convergenceis reached, weight values remain constant duringsubsequent presentations of the entire data set inany order. This measure is independent from com-putational complexity (as defined in the followingsubsection), since an algorithm may require several

epochs to converge using very simple processing, orvice-versa. The product of the two measures pro-vides useful insight into the amount of processingrequired by each SONN to produce its best asymp-totic clustering quality, while sustaining a desiredclustering rate.

3.3. Computational complexity

A first-order approximation of the computa-tional complexity for the SONN algorithms may beobtained by assessing their execution time on anidealized computer. Thus, the time complexity, ¹,combined with a fixed computing area C, allows forcomparison of area-time complexities, C¹. To thateffect, assume that the SONN algorithms are im-plemented as computer programs running ona generic, single processor, random access machine(RAM) [9], where instructions are executed oneafter the other. This generic machine is capable ofno more than one operation per cycle (i.e., neitherVLIW or superscalar). Using the RAM modelavoids the challenging task of accurately determin-ing the performance of specific VLIW or supersca-lar machines, which is beyond the scope of thisanalysis.

Time complexity can be estimated from the max-imum execution time required to process a singleinput pattern. The result is a total worst-case run-ning time formula, ¹, which summarizes the behav-ior of a SONN algorithm as a function of two keyparameters: the dimensionality of the input pat-terns, M, and the number of F2 output neurons, N.Specifically, ¹ can be defined as the sum of theworst-case running times ¹

pfor each operation

p that is required to process an input [9]:

¹"+p

¹p"+

p

op) n

p, (3)

where op

is the constant amount of time needed toexecute an operation p, and n

pis the number of

times this operation is executed.For simplicity, we assume that p can take one out

of two values, 1 or 2, where o1

is the time requiredto execute an elementary operation such as:x#y, x!y, max(x, y), min(x, y), 1!x, x(y, etc.,


and o2is the time needed to compute a division x/y,

a multiplication x ) y, a square root Jx, or anexponent ex. In addition, x and y are assumed to bereal numbers represented by an integer with a b bitresolution, where b corresponds to the number ofbits needed to represent each SONN’s elementaryvalues (i.e., synaptic weights, input pattern ele-ments, and parameters) with a sufficient precision.Operations of type p"2 are more complex thanthose of type p"1, and their complexity, which isa function of b, depends on their specific implemen-tation. Nevertheless, to simplify the analysis, wepresume that o

2KF(b) ) o

1"F ) o

1, where F re-

mains as an explicit parameter in the complexityformulas.

3.4. Data set

The data gathered by DREO consists of 800radar pulses from 12 different shipborne navigationradars. As with any field trial, the recorded signalsshow imperfections in the radar transmitters, andexhibit distortion and noise due to the propagationchannel and the collection equipment. For in-stance, the signal-to-noise ratio varies greatly frompulse to pulse, and from file to file (from 15 to45 dB). This is due to the circular scan of the radars,and their varied power and distance from the col-lection site. Also, one file contains pulses with un-usual characteristics indicative of a radar operatingwith a defective magnetron tube. These pulses werenot removed, for they reflect anomalies sometimesencountered in the field.

After the trial, DREO reduced the dimensional-ity of the data to simplify computer simulation andeventual implementation of the clusterer. Thus, 15real-valued features were extracted from each radarpulse. The feature extraction algorithm has its ownlimitations, and produced a few outliers, whichwere not removed. The final data set was thennormalized using a linear transformation so thatvalues in every dimension range between 0 and 1.

Note that special attention was paid to ensurethat clustering the data set would be difficultenough to fully exercise the algorithms. One shouldtherefore focus on their relative, rather than abso-lute, performance.

The number of pulses per file is nominally 50,but two files contain 100 and 200 pulses, respec-tively. An uneven number of pulses per radar istypical of the application. Indeed, some radarscan transmit in excess of 300,000 pulses/s in theirpulse Doppler mode, and even a few thousandpulses/s in their medium PRI mode. The smallnumber of pulses per file in the data set is a conse-quence of the collection equipment capabilities.Nevertheless, a train of 50 pulses corresponds toa few beam illuminations by a radar in mediumPRI mode, and should be sufficient for ESMdetection.

Fig. 2 shows the result of a two-dimensionalprincipal component analysis (PCA) and linear dis-criminant analysis (LDA) projections for the dataset. Although both PCA and LDA are linear pro-jection methods, LDA benefits from a priori clusterlabel information. Clearly, the clusters cannot bedescribed by the same statistics. Some are notGaussian-distributed, and they sometimes overlapone another.

The radar set described above represents datafrom a stream of pulses collected for one realisticscenario, under which a SONN would be calledupon for automatic deinterleaving. Even thoughthe data set is representative, it is far from compre-hensive: there are several types of radar ESM sys-tems, numerous different theaters of operation, andan ever growing variety of radar sources. Conse-quently, two standard data sets, called Wine Recog-nition and Iris, were used to broaden the scope ofthis research. A brief description, and the simula-tion results obtained on these sets can be found inAppendix A.

3.5. Data presentation order

Since sequential clustering results depend on theorder of presentation of the patterns, three types ofpresentation orders are considered in this compari-son.

A random presentation order consists of randompermutations of the input data set patterns, regard-less of the class.

A burst presentation order is defined by a ran-dom sequence of groups of patterns called bursts,


Fig. 2. Two-dimensional projections for the Radar data set.

where each burst contains patterns from the sameclass. The specific patterns which form the burst,and the class associated with the burst are bothselected at random (from classes with remainingpatterns). The burst size is a random variable witha normal distribution, whose average is equal to5 and whose standard deviation is equal to 3.Bursts of negative size are ignored.

A radar-like interleaved presentation order isobtained as follows. First, the patterns in eachclass are permuted at random. Then, four ficti-tious radar parameters are chosen at randomfor each class. Initial scan time (IS) and scan peri-od (SP) are chosen from a uniform distributionbetween 0 and 1. Beamwidth (BW) and pulse rep-etition interval (PRI) are selected from uniformdistributions between 0 and 0.25 )SP, and between0 and 0.25 )BW, respectively. These parametersallow the computation of a time of arrival (TOA)for every pattern of each class. Sorting the patternsby TOA yields an interleaved, radar-like, order ofpresentation.

4. Comparison results

4.1. Clustering quality and convergence time

For simulation purposes, MATLAB programswere written for FA, FMMC, IAFC, andISODATA algorithms. SOFM simulations weredone using the ‘SOM—PAK’ software.1 Prior toeach simulation, the normalized input patterns ofthe data set were organized according to one of thethree statistical orders (random, burst or inter-leaved). The sequence of patterns was then repeat-edly presented to the algorithms under test untilweights remained stable for two successive epochs,yielding the clustering A. The data set presentationorder was kept constant from one epoch to thenext. After the simulation, the Rand Adjusted and

1This software was obtained via anonymous FTP fromcochlea.hut.fi.


Table 1Summary of best simulation results for the Radar data set

SONN algorithm (parameter values) Performance measures

Clustering quality Convergence time

presentation order SRA

(A, R) SJ(A, R)

Mean St. dev. Mean St. dev. Number of epochs

ISODA¹A (2)¸)4; ∀Nc; 0.1)h

c)0.2; 0.1)h

s)0.5; 14)h

N)22)

random 0.93 0.122 0.99 0.043 I*15burst 0.96 0.124 0.99 0.041 I*15interleaved 0.96 0.163 0.99 0.062 I*15

Fuzzy AR¹ (FA) (0.85)o)0.95; b"1, a"1)random 0.61 0.056 0.93 0.041 3.8burst 0.59 0.109 0.93 0.049 3.2interleaved 0.80 0.130 0.96 0.050 3.6

Fuzzy Min—Max Clustering (FMMC) (c"5; 0.01)h)0.25)random 0.45 0.062 0.92 0.038 2.2burst 0.67 0.064 0.94 0.062 2.0interleaved 0.72 0.099 0.95 0.067 2.0

Integrated Adaptive Fuzzy Clustering (IAFC) (c"1; m"2; 0)p)0.6; 0.1)k)1; 0.5)q)0.6)random 0.62 0.002 0.88 0.001 17.6burst 0.63 0.060 0.89 0.025 20.9interleaved 0.66 0.046 0.90 0.021 21.2

Self-Organizing Feature Mapping (SOFM) (5]5 map; 2)f)16; 0.03)a(0))0.5; 2)r(0))3)random 0.95 0 0.99 0 50burst 0.95 0 0.99 0 50interleaved 0.95 0 0.99 0 50

Jaccard scores were computed, using the refer-ence clustering, R, available for the data. Thescores were stored along with the number ofepochs needed for convergence. The results of 20distinct simulations were combined to extract cen-tral tendencies, and to determine the effect of pre-senting the input patterns in different orders.

Simulation results for the Radar data set aregiven in Table 1. (Similar results obtained usingthe Wine Recognition, and Iris data are given inAppendix A.) The table contains the mean andstandard deviation of the best scores that canbe obtained from the four SONNs and ISODATAfor all three statistical presentation orders. Con-vergence time, and parameter values2 as obtained

by trial and error are also shown. These para-meter values always reflect the least amount ofprocessing needed to produce the best clusteringscore.

For Table 1, the Rand Adjusted and Jaccardscores appear to describe similar trends. ISODATAconfirmed our expectations by yielding thehighest scores for all but one simulation, andthus offers a realistic performance target for theSONNs.

2The reader is referred to the notation used for parameters inthe original papers describing ISODATA, FA, FMMC, IAFC,and SOFM.


Table 2Summary of the computational complexity estimations

Computational complexity

SONN algorithm Worst-case running time (¹/o1) Growth rate

FA 6NM#NF#4MF#3N#5M O(NM)FMMC 3NMF#25NM#2NF!MF#2N!11M O(NM)IAFC 2NMF#NM#7NF#2MF#5N#9F#9 O(NM)SOFM 3N@MF#4N@M#5N@F!MF#6N@!M#3F#3 O(N@M)

SOFM is the SONN that achieves the best over-all scores, or rather score, since this networkproduces consistent clusterings that are the sameacross all presentation orders. This is a direct con-sequence of its slow convergence, combined withthe map interpretation post-processing. This excel-lent score (S

RA(A, R)"0.95) is comparable to those

of ISODATA, but it is obtained after no less than50 epochs. Remember that convergence speed isunder user control, and it was set to yield the bestpossible score.

FA usually scores second best, achieving resultsthat are significantly better with the interleavedpresentation order (S

RA(A, R)"0.80) than with

either the random (SRA

(A, R)"0.61) or the burst(S

RA(A, R)"0.59) presentation orders. These

scores are significantly lower than those of SOFMand ISODATA, but they are obtained after only3 to 4 epochs, indicating that it is a very responsivenetwork.

FMMC converges even faster than FA, and thismay explain that it yields mixed results. Note-worthy is the low Rand Adjusted score(S

RA(A, R)"0.45) obtained for the random order of

presentation, in contrast to the corresponding Jac-card score (S

J(A, R)"0.92), which is not as bad.

Scores for the burst and interleaved orders arecomparable to those of FA.

IAFC takes from about 17 to 22 epochs to con-verge, but yields scores that are no better thanthose of FA and FMMC. Nonetheless, the stan-dard deviation of the scores is low, meaning thatthe results obtained are consistent from one simula-tion to the next.

An attempt was made to improve FA scoresby means of a post-processing mechanism some-

what similar to SOFM’s map interpretationprocedure. Unfortunately, this did not yield higheroverall scores, and thus the results are not shownin Table 1. It did, however, allow good scoresto be obtained over a broader range of vigilanceparameter values.

The results in Table 1 clearly indicate that thedata presentation order has a significant impact onperformance. With the exception of SOFM, thealgorithms usually perform better as the input pre-sentation order moves from the completely randomcase to the more structured ones, where patternsfrom a category may be presented together. Thestandard deviation of the scores are also dependenton the data presentation order. For instance, withFA and FMMC, the scores often vary more for theinterleaved than for the burst, and more for theburst than for the random orders. In most cases,high scores appear to be correlated with high vari-ance for the burst and interleaved presentationorders. With these orders, sequences are defined bymixtures of bursts of patterns from the same cluster.If we accept that scores depend on the size of thebursts, then the randomly defined burst sizes mayexplain the high variance of the scores. Burst sizeshave no notable effect on convergence time.

4.2. Computational complexity

For brevity, only the final results of our analysisare shown here. The reader is referred to Appen-dix B for further details. Table 2 shows the totalworst-case running times per input pattern ina normalized format (¹/o

1), when parameter F is

the same for all operations of type p"2. The


Fig. 3. Worst-case running time (¹/o1) versus N, assuming that M"16 and F"10. The number of map neurons in SOFM is set to

N@"3N.

corresponding growth rates, valid when theparameters M and NA1, and the parameter F isconstant, are also given. It turns out that the com-plexity of all SONNs is described by the sameasymptotic growth rate, O(NM).

Overall, the results in Table 2 indicate that FA’sworst-case processing is the least sensitive to theparameters M, N and F. Its processing time isconsumed mainly by the winner-take-all competi-tions between output neurons (choice functioncomputations). Among the other networks, IAFChas the second lowest worst-case running time perinput pattern, followed by SOFM and FMMC.Although FA, FMMC, and IAFC have similarART-type control structures, FMMC incurs anoverlap test and contraction procedure to eliminatehyperrectangle overlap, whereas IAFC incurs twomembership measures (fuzzy and intra-cluster), anda supplementary category choice procedure basedon Euclidean distances. SOFM processing is morestraightforward than that of ART-type SONNs,since there is no category expansion test, nor search

process. The selection of a winning map neuron hasa complexity comparable to FMMC and IAFCchoice functions, even though the number of mapneurons, N@, is usually greater than N (several mapneurons may be needed to represent a single cat-egory with SOFM). However, the weight updatestep may involve several map neurons, instead ofonly one with ART-type SONNs. This m-learnstrategy is compounded by the map interpretationpost-processing.

The worst-case processing time as a function ofthe number of neurons N is an important consid-eration for implementation. As an example, sup-pose that one wishes to implement a SONN havingM"16 dimensions using a factor F"10. Fig. 3shows ¹/o

1versus N for all four SONNs. The

number of map neurons in SOFM is assumed to bethree times the number of categories (N@"3N). FAis clearly the fastest to process an input, followed byIAFC, SOFM and FMMC.

Another consideration is storage of weightvalues. FA and FMMC require 2MNb bits for


storage of their weights. IAFC needs only half ofthat memory when input patterns are normalized.A memory size of N@Mb bits is required withSOFM, but N@ is normally greater than N (we haveassumed a ratio of 3 to 1 so far).

Although this complexity analysis is based ona generic single processor random access machine,all these networks are suitable for parallel imple-mentation using a multitude of VLSI architectures.The time complexity estimations obtained in ouranalysis then provide an indication of the semicon-ductor area required by the parallel hardware tomeet a desired input pattern rate.

4.3. Discussion

Simulation results on Radar data show thatSOFM can yield excellent clustering scores if it isgranted sufficient convergence time. By contrast,FA converges much faster, is computationally inex-pensive, but yields lower scores than SOFM. In thecase of FMMC and IAFC, the clustering quality iscomparable to FA, but their processing require-ments are significantly higher. These fundamentaldifferences suggest that SOFM and FA could beattractive alternatives suitable for different radarESM applications.

Consider the following example. The highestclustering score for the Radar data presented inthe interleaved order is obtained using SOFM(S

RA(A, R)"0.95). This is a very satisfactory score

for radar ESM. However, SOFM converges veryslowly and is computationally intensive. AssumingM"16, F"10 and N@"3N"30 map neurons, itrequires approximately 18 000 elementary opera-tions (¹/o

1) for every input pattern. FA, on the

other hand, achieves the second highest score forthe interleaved radar data (S

RA(A, R)"0.80). This

is significantly lower than that of SOFM. But, as-suming M"16, F"10 and N"10 output neur-ons, FA requires just 2500 elementary operationsper input pattern, and converges with almost 14times fewer input pattern presentations thanSOFM (3.6 epochs instead of 50).

Based on such numbers, SOFM could be wellsuited for radar ESM systems employed for long-range surveillance, intelligence, or targeting. In

these tasks, accuracy prevails over timing: somedelay can be tolerated in exchange for enhancedprecision. Also, the high computational cost ofSOFM may be less of a problem in shipborne orland-based installations, where equipment of someweight and size can be accommodated.

As for FA, it could better address the require-ments of radar ESM systems used for threat alert.In such systems, reaction time must be minimizedin order to engage timely protection measuresagainst missiles, anti-aircraft artillery, interceptaircraft and other threats. The fast convergence ofFA would thus be an indisputable asset. Also,the low computational complexity of FA wouldpermit its use where space is scarce, and cost mustremain low, like in small aircraft radar warningreceivers.

5. Conclusions

A comparison of four self-organizing neural net-works (SONNs) that are potentially suitable forhigh throughput clustering of radar pulses in ESMsystems has been presented. These SONNs are theFuzzy Adaptive Resonance Theory (FA), FuzzyMin—Max Clustering (FMMC), Integrated Adap-tive Fuzzy Clustering (IAFC), and Self-OrganizingFeature Mapping (SOFM).

The SONNs have been compared from threestandpoints — clustering quality, convergence time,and computational complexity. Computer simula-tions have been fed with a Radar data set pre-sented in several statistical orders. Convergencetime has been measured by counting successivepresentations of the data set prior to weight stabil-ization. Clustering quality has been assessedusing the Rand Adjusted and Jaccard similaritymeasures.

Simulation results have shown that: (1) SOFMusually yields the best scores followed by FA,FMMC, and IAFC, (2) FMMC and FA convergethe fastest, followed by IAFC and SOFM, (3) thedata presentation order has a significant impact onthe scores of ART-type networks, and their varia-bility, and (4) SOFM, when granted a long conver-gence time, is insensitive to the data presentationorder. The SONNs have then been studied from an


algorithmic perspective to estimate worst-case run-ning times, and consequently, computational com-plexity. Of the four SONNs, FA has been shown tohave the lowest complexity, followed by IAFC,SOFM, and FMMC.

SOFM and FA emerge as suitable candidates forsorting pulses in radar ESM, albeit for differentreasons. On the one hand, SOFM can achieveexcellent clustering scores at the expense of a highcomplexity and potentially long convergence time.It would therefore be well suited for long rangesurveillance, intelligence and targeting. On theother hand, FA has the potential be to fast enoughfor use in threat alert systems. The selection ofeither SONN would ultimately depend on the spe-cifics of the ESM application, and effect a tradeoffbetween clustering accuracy and computationalefficiency.

Notation

a an input patternM dimensionality of input patterns.N, N@ number of output layer neurons in the

three ART-type SONNs, and inSOFM’s 2D map, respectively.

i, j indices denoting neurons in the input,and the output layers, respectively.

bj, u

j, �

j, w

jprototype weight vectors for theSONN output layer neurons j, wherej"1, 2,2, N or N@.

F1, F2 SONN input layer, and output layers,respectively.

Bj

FMMC’s fuzzy set representation ofcategory j.

A, B partition of a data set’s n patterns intoK groups (a clustering). Further, A isused to describe a clustering producedduring a computer simulation.

R a data set’s reference clustering.ah

category label assigned to pattern h ofa data set.

c11

, c12

,c21

, c22

four elements of the contingency table.

m total number of combinations of twoout of n patterns.

S(A, R) measure of similarity between twoclusterings, A and R. It is noted

SJ(A, R), for the Jaccard measure,

and SRA

(A, R) for the Rand adjustedmeasure.

¹ maximum execution time required toprocess an input pattern (an estima-tion of the time complexity).

C computing area.p, q indices denoting different types of op-

erations considered in the complexityanalysis.

op

cost of an operation of type p.np

number of times operation p is ex-ecuted.

x, y real numbers represented by an inte-ger with a b bit resolution.

b number of bits needed to representa SONN’s elementary values.

F parameter used to reflect the use ofhigher complexity operations in worst-case running time polynomials.

a FA’s choice parameter.3o FA’s vigilance parameter.b FA’s learning rate parameter.c FMMC’s sensitivity parameter. In ad-

dition, with IAFC it is a factor used tocontrol the shape of clusters formed.

h FMMC’s maximum hyperrectanglesize parameter.

m weight exponent used to computeIAFC’s fuzzy membership values.

p IAFC’s fuzzy membership test para-meter.

j IAFC’s learning rate parameter.k parameter that regulates the speed at

which j decreases with the number ofdata set iterations.

r(t) SOFM’s neighborhood radius para-meter at time t.

a(t) SOFM’s learning rate parameter attime t.

f SOFM’s category center parameter(i.e., the minimum number of assignedinputs required for a neuron to beinterpreted as an actual category).

3The remaining symbols are parameters taken directly fromthe original papers that describe FA, FMMC, IAFC, andSOFM.


Acknowledgements

This work was supported in part by the DefenceResearch Establishment Ottawa, the CanadianMicroelectronics Corporation, le Fonds pour laFormation de Chercheurs et l’Aide a la Recherche(province of Quebec), and the Natural Sciences andEngineering Research Council of Canada.

Appendix A. Simulation results for the WineRecognition and Iris data sets

The ¼ine Recognition data set4 is the result ofa chemical analysis of wines from three differentcultivars. The problems consists is determining theorigins of 178 wine samples, where 13 attributescharacterize the constituents found in each sample.The well-known Iris data set [12] contains 150flowers belonging to three species of iris flowers:satosa, versicolor and virginica. Each species con-tains 50 flowers that are characterized by 4 features:petal and sepal length and width.

Simulation results for the Wine Recognition andIris data sets are given in Tables 3 and 4, respec-tively. As with the Radar data, SOFM most oftenachieves the highest SONN scores, usually fol-lowed by FA. The only exceptions are found whenthe Wine and Iris data are presented in the inter-leaved order, in which case FA scores the best.IAFC performs almost as well as FMMC for theWine data, and surpasses FMMC for the Iris data.In this last case, IAFC almost ties FA for secondbest. Overall, IAFC appears to be more sensitivethan the others the the distribution of the patternsin input space.

It is worth noting that the gap between SOFMscores and those of the other three SONNs is not aswide as with the radar data. Besides, the threeART-type networks usually perform better whendata are presented in more structured, radar-likeorders. This holds true to the extent that FA scoresslightly higher than ISODATA for Iris data pre-sented in the interleaved order.

4This set was obtained from the UCI Machine LearningRepository (www.uci.edu/&mlearn).

The SONN convergence times appear to vary ina proportional manner from one data set to thenext: FMMC always converges with the leastamount of epochs, followed closely by FA, then byIAFC and SOFM. On average, FA and FMMCconverge slightly faster for the Iris data than withthe Wine data, and faster with the Wine data thanthe Radar data. IAFC and SOFM converge inmuch fewer epochs with the Wine data than eitherthe Iris or Radar data. This discrepancy may be dueto a sensitivity to data structures with overlappingclusters. This is especially true for IAFC, whoselearning rate, j, scales according to fuzzy andintra-cluster membership functions [21]. Noticethat its convergence for the Iris data requires moreepochs than for the Radar data set. In any case,there seems to be little correlation between conver-gence time and presentation order.

Appendix B. Analysis of computational complexity

In this appendix, details of a computational com-plexity analysis are presented5 for each SONNalgorithm. First, the maximum time required toexecute every operation of type p is derived(¹

p"o

p) n

p). Table 5 gives a breakdown of time

complexities, organized according to algorithmsteps. The time complexity of a step is equal to theproduct of its cost per iteration and the number ofiterations it needs. Notice how the values for someof these steps are obtained by adding the contribu-tion of several sub-steps (shown directly below).Then, the sum of these time complexities yieldsa total worst-case running time per input pattern,¹"+

p¹

p, and a corresponding growth rate. The

resulting complexity polynomial is expressed ina normalized format, ¹/o

1, in terms of network

parameters M, N and F, and provides an estimate

5A similar type of analysis has been conducted by Lawrenceet al. [25] to approximate the computational complexity ofvarious tasks done by a face recognition system. This type ofanalysis is important in the context of implementing real-timesystems, where complexity scales according to system para-meters.


Table 3Summary of best simulation results for the Wine Recognition data set




(A, R) SJ(A, R)


ISODA¹A (¸"1; Nc*5; 0.3)h

c)0.6; 0.1)h

s)0.4; h

N*8)


Fuzzy AR¹ (FA) (b"1; a"1; 0.35)o)0.5)random 0.26 0.056 0.68 0.013 2.9burst 0.29 0.077 0.70 0.020 3.0interleaved 0.49 0.110 0.77 0.031 2.8

Fuzzy Min—Max Clustering (FMMC) (4)c)5; 0.6)h)0.75)random 0.07 0.012 0.67 0.001 2.1burst 0.18 0.013 0.69 0.004 2.0interleaved 0.32 0.112 0.71 0.031 2.0

Integrated Adaptive Fuzzy Clustering (IAFC) (c"1; m"2; 0.6)p)0.9; 0.1)k)0.8; 0.4)q)0.6)random 0.13 0.013 0.68 0.003 6.5burst 0.13 0.009 0.68 0.002 7.6interleaved 0.18 0.009 0.70 0.002 6.8

Self-Organizing Feature Mapping (SOFM) (3]3 map; 2)f)11; 0.01)a(0))1; 1)r(0))2)random 0.48 0 0.73 0 14burst 0.50 0 0.73 0 14interleaved 0.50 0 0.73 0 14

of computational complexity. The following costvalues o

pare used throughout the analysis:

f o1: the time required to execute elementary, low-

level operations (i.e., x#y, x!y, max(x, y),1!x, x(y, etc.). The values x and y are realnumbers represented with a b bit resolution.

f o2,q

: the time needed to compute a division x/y(o

2,1), multiplication x ) y (o

2,2), a square root Jx

(o2,3

), or an exponent ex (o2,4

). Notice that opera-tions of type p"2 are subdivided into opera-tions of type q"M1,2,3,4N on account for theirdissimilar complexities. The complexity of theseoperations is a function of b (o

2,q"o

2,q(b)),

and they differ according to specific implemen-tations. However, in order to ease the analysisand the comparison of results, we assume that

o2,1

Ko2,2

Ko2,3

Ko2,4

KF (b ) ) o1"F ) o

1,

where parameter F reflects the use of highercomplexity operations in the analysis. This isa reasonable assumption if dedicated hardware isavailable to support the requested operations.

As explained previously, this complexity analysisassumes that the SONN algorithms are imple-mented using a random access machine (RAM)model of computation [9]. They are also supposedto be executed in an order permitting efficient reuseof data (e.g., no values are computed twice). Inputpatterns are considered to be normalized, and allweight and parameter values are presumed to beinitialized prior to the presentation of any data.The time associated with these last two operationsis ignored since it is comparable for all the SONNs.


Table 4Summary of best simulation results for the Iris data set




(A, R) SJ(A, R)


ISODA¹A (1)¸); ∀Nc; 1)h

c)1.5; h

s(1.2; 15)h

N)22)


Fuzzy AR¹ (FA) (0.7)o)0.75; a"1; b"1)random 0.68 0.077 0.82 0.042 2.3burst 0.68 0.085 0.83 0.047 2.6interleaved 0.75 0.109 0.86 0.042 1.8

Fuzzy Min—Max Clustering (FMMC) (4)c)5; 0.4)c)0.55)random 0.53 0.078 0.78 0.034 1.8burst 0.60 0.103 0.79 0.036 2.1interleaved 0.68 0.218 0.82 0.045 1.6

Integrated Adaptive Fuzzy Clustering (IAFC) (c"1; m"2; 0.2)p)0.5; 0.3)k)0.7; 0.8)q)1)random 0.68 0.046 0.81 0.019 26.3burst 0.68 0.016 0.83 0.009 24.7interleaved 0.72 0.033 0.84 0.012 23.0

Self-Organizing Feature Mapping (SOFM) (3]3 map; 7)f)12; 0.07)a(0))0.3; 1)r(0))3)random 0.69 0 0.84 0 45burst 0.69 0 0.84 0 45interleaved 0.69 0 0.84 0 45

The SONNs form clusters progressively by sequen-tial processing of input data patterns. ISODATA isan iterative procedure that requires all the inputdata at once to form a predefined number of clus-ters. Since ISODATA cannot be analyzed in thesame way as the SONNs, it is omitted from thecomplexity analysis.

B.1. Fuzzy ART (FA)

In Table 5, the break down of time complexitiesshows that operation B, the computation of theoutput neuron choice functions, is potentially themost costly with FA. The sequence of the FAsearch process (operation C) has been modified

according to Section 2.2. The vigilance test is com-puted only once for all N output neurons prior tocategory choice. Similar modifications to FMMCand IAFC were also assumed. The total worst-caserunning time ¹

FAcorresponds to the sum of the ele-

ments multiplied in the ‘time complexity’ column:

¹FA

"o1M#N[o

1(6M#1)#o

2,1]

#2o1N#4M(o

1#o

2,2)

"NM(6o1)#N(3o

1#o

2,1)

#M(5o1#4o

2,2)

"o1(6NM#3N#5M)#o

2,1(N)

#o2,2

(4M).


Table 5Breakdown of time complexities

SONN algorithm Time complexity

Steps of the algorithm Cost per iteration Number of iterations

Fuzzy Art (FA)A. Complement coding o

1M 1

B. Choice functions o1(6M#1)#o

2,1N

C. Search process:C.1. Vigilance test o

1N

C.2. Category choice o1

ND. Prototype vector update 4M(o

1#o

2,2) 1

Fuzzy Min—Max Clustring (FMMC)A. Membership functions M(9o

1#2o

2,2)#o

2,1N

B. Search process:B.1. Expansion test 3o

1M#o

1#o

2,1N

B.2. Hyperrectangle choice o1

NC. Hyperrectangle expansion 2o

1M 1

D. Overlap test 12o1M N!1

E. Hyperrectangle contraction M(o1#o

2,1) N!1

Integrated Adaptive Fuzzy Clustering (IAFC)A. Pre-computations

A.1. DDa!�jDD, ∀j M(o

1#o

2,1)#o

2,3N

A.2. 1/ DDa!�jDD2, ∀j o

2,1#o

2,2N

A.3. +Nj/1

1/ DDa!�jDD2 o

1N

B. Choice functions o2,2

M NC. Search process:

C.1. Vigilance test 2o1#o

2,1#o

2,2#o

2,4N

C.2. Membership test o1#o

2,1N

C.3. Category choice o1

ND. Centroid vector update 2o

2,2M#9o

1#3o

2,1#6o

2,21

Self-Organizing Feature Mapping (SOFM)A. Map neuron selection:

A.1. Euclidean distances M(o1#o

2,1)#o

2,3N@

A.2. Best-match choice o1

[email protected]. Increment its input count o

1N@

B. Update prototype(s):B.1. Define neighborhood 3o

1#2o

2,1#o

2,3N@

B.2. Adjust weights M(2o1#o

2,2) N@

C. Modify a(t) and r(t) 2(o1#o

2,1#o

2,2) 1

D. Map interpretation:D.1. Find category centers o

1N@

D.2. Relate neurons M(o1#o

2,1)#o

2,3N@!1

D.3. Category choice o1

1


If parameter F is comparable for all type p"2operations, o

2,1Ko

2,2KF ) o

1, then

¹FA

"o1) [6NM#NF#4MF#3N#5M].

(B.1)

The rate of growth of ¹FA

when F is constant M,NA1 is O(NM).

B.2. Fuzzy Min—Max Clustering (FMMC)

As with FA, the computation of output neuronmembership functions (operation A) is potentiallyvery demanding. FMMC’s simple hyperrectangleexpansion procedure (operation C) is equivalent toFA’s fast learning rule (with b"1). However, itscombination of overlap test and contraction pro-cedure (operations D and E), are also notably timeconsuming. The total worst-case running time¹

FMMCis

¹FMMC

"N[M(9o1#2o

2,2)#o

2,1]

#N[3o1M#2o

1#o

2,1]#2o

1M

#(N!1)12o1M#(N!1)[M(o

1#o

2,1)]

"NM(25o1#o

2,1#2o

2,2)

#2N(o1#o

2,1)!M(11o

1#o

2,1)

"o1(25NM#2N!11M)

#o2,1

(NM#2N!M)#o2,2

(2NM).


2,1Ko

2,2KFo

1, then

¹FMMC

"o1) [3NMF#25NM#2NF

!MF#2N!11M]. (B.2)

The rate of growth of ¹FMMC

when F is constantand M, NA1 is O(NM).

B.3. Integrated Adaptive Fuzzy Clustering (IAFC)

The computation of the Euclidean distances andrelated values in operation A (between input a and

the prototype vectors �j) is potentially very time

consuming. These values are needed at several in-stances in IAFC’s processing. For example, fuzzymembership values are needed for an alternatechoice procedure (see operation C.2). The complex-ity required for the choice functions and the searchprocess is comparable to that of FMMC. Its cen-troid update (operation D) is as costly as the proto-type vector update of FA. The total worst-caserunning time ¹

IAFCis

¹IAFC

"N[M(o1#o

2,1)#o

1#o

2,1#o

2,2#o

2,3]

#N[o2,2

M]#N[4o1#2o

2,1#o

2,2

#o2,4

]#2o2,2

M#9o1#3o

2,1#6o

2,2

"NM(o1#o

2,1#o

2,2)#N(5o

1#3o

2,1

#2o2,2

#o2,3

#o2,4

)#M(2o2,2

)

#9o1#3o

2,1#6o

2,2

"o1(NM#5N#9)#o

2,1(NM#3N#3)

#o2,2

(NM#2N#2M#6)

#o2,3

(N)#o2,4

(N) .


2,1Ko

2,2Ko

2,3Ko

2,4KF ) o

1, then

¹IAFC

"o1) [2NMF#NM#7NF#2MF

#5N#9F#9] . (B.3)

The rate of growth ¹IAFC

when F is constant andM, NA1 is O(NM).

B.4. Self-Organizing Feature Mapping (SOFM)

The SOFM processing is more straightforwardthan that of the ART-type SONNs, since there is nosearch process. Most of the workload is concen-trated in three operations: A, B and D. The selec-tion of a winning map neuron has a complexitycomparable to the FMMC and IAFC choice func-tions, even though N@ is usually greater than N (sev-eral map neurons may be needed to representa single category with SOFM). Notice also the


weight update required for several map neurons,instead of just one with ART-type SONNs. Thetotal worst-case running time ¹

SOFMis

¹SOFM

"N@[M(o1#o

2,1)#2o

1#o

2,3]

#N@[M(2o1#o

2,2)#3o

1#2o

2,1#o

2,3]

#2(o1#o

2,1#o

2,2)#N@[M(o

1#o

2,1)

#o1#o

2,3]!M(o

1#o

2,1)#o

1!o

2,3

"N@M(4o1#2o

2,1#o

2,2)

#N@(6o1#2o

2,1#3o

2,3)!M(o

1#o

2,1)

#3o1#2o

2,1#2o

2,2!o

2,3

"o1(4N@M#6N@!M#3)

#o2,1

(2N@M#2N@!M#2)

#o2,2

(N@M#2)#o2,3

3N@!1).


2,1Ko

2,2Ko

2,3KF ) o

1, then

¹SOFM

"o1) [3N@MF#4N@M#5N@F!MF

#6N@!M#3F#3]. (B.4)

The rate of growth of ¹SOFM

when F is constant andM, N@A1 is O(N@M).

References

[1] M.R. Anderberg, Cluster Analysis for Applications, Aca-demic Press, New York, 1973.

[2] J.A. Anderson, M.T. Gately, P.A. Penz, D.R. Collins,Radar signal categorization using a neural network, Proc.IEEE 78 (10) (1990) 1646—1656.

[3] G.H. Ball, D.J. Hall, ISODATA, an iterative method ofmultivariate analysis and pattern classification, Proc.IFIPS Congress, 1965.

[4] Behrooz Kamgar-Parsi, Behzad Kamgar-Parsi, J.C. Scior-tino Jr., Automatic data sorting using neural networktechniques, Naval Research Laboratory ReportNRL/FR/5720-96-9803, February 1996.

[5] J.C. Bezdek, Pattern Recognition with Fuzzy ObjectiveFunction Algorithms, Plenum Press, New York, 1987.

[6] J.C. Bezdek, Some non-standard clustering algorithms, in:P. Legendre, L. Legendre (Eds.), NATO ASI Series, Vol.G14, Developments in Numerical Ecology, Springer, Ber-lin, 1987.

[7] G.A. Carpenter, S. Grossberg, A massively parallel archi-tecture for a self-organizing neural pattern recognitionmachine, Comput. Vision Graphics Image Process. 37(1987) 54—115.

[8] G.A. Carpenter, S. Grossberg, D.B. Rosen, Fuzzy ART:Fast stable learning and categorization of analog patternsby an adaptive resonance system, Neural Networks 4 (6)(1991) 759—771.

[9] T.H. Corman, C.E. Leiserson, R.L. Rivest, Introduction toAlgorithms, MIT Press, New York, 1990.

[10] R.C. Dubes, A.K. Jain, Algorithms for Clustering Data,Prentice-Hall, Englewood Cliffs, NJ, 1988.

[11] R.O. Duda, P.E. Hart, Pattern Classification and SceneAnalysis, Wiley, New York, 1973.

[12] R.A. Fisher, The use of multiple measurements in taxo-nomic problems, Ann. Eugenics 7 (1936) 179—188.

[13] P.M. Grant, J.H. Collins, Introduction to electronic war-fare, Proc. IEE 129, Part. F (3) (June 1982) 113—129.

[14] S. Grossberg, Adaptive pattern classification and universalrecoding: I. Parallel development and coding of neuraldetectors, Biol. Cybernet. 23 (1976) 121—134.

[15] S. Grossberg, Adaptive pattern classification and universalrecoding: II. Feedback, oscillation, olfaction, and illusions,Biol. Cybernet. 23 (1976) 187—207.

[16] S. Haykin, Neural Networks: A Comprehensive Founda-tion, Macmillan College Publishing Co., New York, 1994.

[17] L. Hubert, P. Arabie, Comparing partitions, J. Classifica-tion 2 (1985) 193—218.

[18] S. Keyvan, L.C. Rabelo, A. Malkani, Nuclear reactorcondition monitoring by adaptive resonance theory,Proc. IJCNN’92, Baltimore, USA, Vol. 3, 1992, pp.321—328.

[19] Y.S. Kim, S. Mitra, A comparative study of the perfor-mance of fuzzy ART-type clustering algorithms in pat-tern recognition, Proc. SPIE: Intelligent Robots andComputer Vision XI, Boston, USA, November 1992,pp. 335—341.

[20] Y.S. Kim, S. Mitra, Integrated adaptive fuzzy clustering(IAFC) algorithm, Proc. 2nd Fuzz-IEEE, 1993, pp.1264—1268.

[21] Y.S. Kim, S. Mitra, An adaptive integrated fuzzy clusteringmodel for pattern recognition, Fuzzy Sets and Systems,(65) (1994) 297—310.

[22] T. Kohonen, Self-organizing formation of topologicallycorrect feature maps, Biol. Cybernet. 43 (1982) 59—69.

[23] T. Kohonen, Self-Organization and Associative Memory,3rd ed., Springer, Berlin, 1989.

[24] B. Kosko, Neural Networks and Fuzzy Systems: A Dy-namical Approach to Machine Intelligence, Prentice-Hall,Englewood Cliffs, NJ, 1992.

[25] S. Lawrence, C.L. Giles, A.C. Tsoi, A.D. Back, Face recog-nition: A convolutional neural-network approach, IEEETrans. Neural Networks 8 (1) (1997) 98—113.

[26] J.P.Y. Lee, Wideband I/Q demodulators: Measurementtechnique and matching characteristics, Proc. IEE: Radar,Sonar, Navigation, Vol. 143, No. 5, October 1996, pp.300—306.


[27] C.-T. Lin, C.S. George Lee, Neural Fuzzy Systems:A Neuro-fuzzy Synergism to Intelligent Systems, Pren-tice-Hall, Englewood Cliffs, NJ, 1995.

[28] G.W. Milligan, S.C. Soon, L.M. Sokol, The effect of clustersize, dimensionality, and the number of clusters on recov-ery of true cluster structure, IEEE Trans. Pattern Anal.Machine Intelligence 5 (1983) 40—47.

[29] F. Murtagh, Interpreting the Kohonen self-organizing fea-ture map using contiguity-constrained clustering, PatternRecognition Lett. 16 (1995) 399—408.

[30] R. Peper, B. Zhang, H. Noda, A comparative study of theART2-Aand the self-organizing feature map, Proc. IJCNN’93,Nagoya, Japan, October 1993, Vol. 2, pp. 1425—1429.

[31] E. Pesonen, M. Eskelinen, M. Juhola, Comparison of dif-ferent neural network algorithms in the diagnosis of acuteappendicitis, Internat. J. Bio-Med. Comput. 40 (3) (1996)227—233.

[32] D.E. Rumelhart, D. Zipser, Feature discovery by competi-tive learning, Cognitive Sci. 9 (1985) 75—112.

[33] D.C. Schleher, Introduction to Electronic Warfare, ArtechHouse, Dedham, MA, 1986.

[34] P. Simpson, Fuzzy min—max neural networks — Part 2:Clustering, IEEE Trans. Fuzzy Systems 1 (1) (1993)32—45.

[35] J.T. Tou, R.C. Gonzalez, Pattern Recognition Principles,Addison-Wesley, Reading, MA, 1974.

[36] C. von der Malsberg, Self-organization of orientation sen-sitive cells of the striate cortex, Kybernetik 14 (1973)85—100.

[37] Z. Wang, A. Guerriero, M. De Sario, Comparison of sev-eral approaches for the segmentation of texture images,Proc. SPIE: The Internat. Society of Optical Engineering,Vol. 2424, 1995, pp. 580—591.

[38] C.-D. Wann, C.A. Thomopoulos, Comparative study ofself-organizing neural networks, Proc. IWANN’93, Sitges,Spain, June 1993, pp. 316—321.

[39] C.L. Davies, P. Hollands, Automatic processing for ESM,Proc. IEEE 129, Part F (3) (June 1982) 164—171.


Documents

A comparison of self-organizing neural networks for fast clustering of radar pulses