Multi-classifier Fusion Approach based on DataClustering for Analog Circuits Fault Diagnosis

Guoming Song *, Houjun Wang, Hong Liu, and Shuyan Jiang

Abstract - When there are large amount offault classesin analog circuits, normally single multi-class classifiercannot achieve satisfactory diagnosis accuracy because of itsdifficult training process. A method of multi-classifier fusiondiagnosis approach based on data clustering is presented inthis paper to improve fault diagnosis veracity. After extractingfault feature vectors by wavelet transform, fuzzy C-meanclustering algorithm is used to pre-partition the feature spaceinto multiple sub-class groups as binary tree. According to thestructure ofthe fault tree, multi-classifiers are created to formhierarchical diagnosis system. Simulation experimentsdemonstrate that the proposed approach for analog circuitfault diagnosis is superior to conventional ones. The faultdiagnosis accuracy is greater than 98%. It has goodperformance in tackling large number of fault classes inanalog circuits. 1

Index Terms - fault diagnosis, analog circuits, wavelettransform, FCM clustering, fusion.

I. INTRODUCTION

As the integration and complexity of VLSI increasedrapidly, fault diagnosis and fault location are more difficult.According to collected statistics, faults usually occur in analogcircuits for a mixed signal electronic system. In an electronicdevice, although 80 percent of the circuit is digital, about 80percent of the faults or faulty component occur in the analogsegment of the circuit [1]. Some inherent characteristicscaused slow progress on research of analog circuit faultdiagnosis, such as lack of testable information, componenttolerance, poor fault model and circuit nonlinearity etc.Therefore, analog circuit fault diagnosis remains challengingfor researchers. Some techniques of artificial intelligence wereapplied for effectively identifying circuit faults in recentdecades, especially neural networks(NNs) and support vectormachines(SVMs)[2]-[7].To discriminate different fault classes,a certain number of fault features are extracted to traincorresponding classifiers in these approaches, which are usedfor fault diagnosis. Single multi-class classifiers such as NNshow good performance when there are less numbers of faultclasses or different fault classes are isolated enough. However,the fault features of some faults may have high similarity,sometimes they are even overlapped. On the other hand,

IThis work was supported in part by Defense foundation scientific research

fund under Grant No.A1420061264, NNSF under Grant No. 60673011Guoming Song, Houjun Wang, Hong Liu, and Shuyan Jiang are with

School of Automation Engineering, University of Electronic Science andTechnology ofChina,Chengdu,610054 (e-mail: guomings@hotmail.com).

Guoming Song is with Department of Computer Engineering, ChengduElectromechanical College, Chengdu, 610031, China.

978-1-4244-3870-9/09/$25.00 ©2009 IEEE

increasing number of fault class requires higher dimension offeature data and optimal algorithms to gain higher diagnosisprecision. In this scenario, traditional classifiers have to bedesigned with more complicated structure, especially for highdimension data. This will bring difficult training process andhigh misclassification. Some schemes are tried to solveaforementioned problems. For instance, researchers definedfault classes that overlap each other into the same fault classcalled ambiguous group to diminish fault class number andimprove classification capability [8][9]. But this method wasteinformation of feature samples which can distinguish theirattached fault classes from others in the same class groups.Furthermore, they are based on the observation ofexperimental results, not from the point of theoretical criteria.To overcome above problems, a multi-classifier fusionapproach based on data clustering is proposed in this paper.Fuzzy C-Means algorithm was used to divide fault featurespace into fault groups as binary tree, which further decideframe of fault diagnosis system composed of multi-levelclassifiers. Classifiers are designed independently foridentifying respective fault class groups, which is comprisedof small number of fault classes for easier classification. Indiagnosis phase, proper classifier is chosen for recognized thetesting features according to the membership grade to the treenodes. As wavelet transform offers obvious advantages infaulty feature extraction comparing with traditional methodsof signal analysis, wavelet coefficients of output voltagesignals were acquired to form faulty feature vectors.Experimental simulation validates the excellent performanceof our method for a bandpass filter circuit.

II. FAULT DIAGNOSIS APPROACH OF FCM MULTI­

CLASSIFIER FUSION

A. Fuzzy C-mean Clustering

Consider a set of N data vectors X = {Xl' X2 ' • • •, XN } can be

divided into m classes Cl' C2 , •••, Cm ' there exists m x N

matrix U(X) =LUij ]mxN .i =1,2, ...m, j =1,2...N in which

f.1ij indicates the membership grade Xi to class Cj .Fuzzy

clustering matrix U has following properties:(1) Vf.1ij E [0,1] ;

m

(2) LPij =1,Vj ;i=l

(3) 0 < Illii < N, Vi .i=l

FCM algorithm calculates fuzzy classification matrix U andclustering center matrix V which is subjected to constrain that

1217

makes the goal function J reach minimum. J is expressed as: Input feature vectors

m

Equation (1) can be solved with constrain I flij =1 . Thej=1

be>!) is a fuzzy parameter, d(xj,v)=IIXj-vjI12 is the

distance between Xi and Vj which represents their dissimilarity.

B. Hierarchical Fault Diagnosis system ofMuIti- ClassifiersFusion

In conventional diagnosis system, a multi-class classifier isused for identifying potential fault categories and no faultstatus which are usually defined carefully beforehand in acircuit under test. In fact, it is crucial for us to know if thecircuits are healthy or faulty. The discrimination of the twostates can be completed by a binary-class classifier with lesstime-consuming. If the circuit is faulty, some other classifiersare considered to find which fault it occurs exactly. As acomplex fault class set can be divided into groups with smallnumber of fault classes, that means multiple classifiers withsimple structures may provide fault diagnosis with highefficiency. The key is how to pre-classify large number offaulty classes into small groups according to some criteria.

As a widely used data clustering algorithm, Fuzzy C-Meanis suitable for separating the fault feature vectors into probablefault class groups which may include two or more fault classes.In this paper, FCM is performed repeatedly for two-classclustering. After data clustering, the fault feature vector spacewill be expressed as a binary tree in which each fault classbelong to certain tree node or branch. Then multiple classifiersare created to identify corresponding fault class groups. Theframe of hierarchical diagnosis system is showed as figure 1.Because each fault group has small fault classes, it bringseasier training process for respective classifier and gainingbetter diagnosis accuracy.

Fault free ( )Output: 0

1.0..- ---...1

Fig.l Fusion fault diagnosis system of multiple level classifiers

C. Fault Diagnosis Algorithm

Suppose potential fault categories in the analog circuit are

K, Apxn is fault feature vector matrix with p fault patterns

having n dimension of feature. This matrix is used forclassifying and clustering. Fault diagnosis algorithm based ondata clustering and multi-classifier fusion is summarized asfollowing:

Step 1. Input feature patterns under fault free and faultystates to the first level binary-class classifier (classifier 1) fortraining, record the trained parameters. Seek out featurepatterns under fault free condition and keep the faulty featuresfor following FCM clustering to generate binary tree of faultgroups.

Step2. Set initial clustering number C=2, select clustering

center V j from feature matrix randomly.

Step3. Group training patterns into two classes according toiterative formula of data clustering. Label as father class andmother class with respective code.

Step4. Compute clustering center distance and membershipgrade between each feature pattern to father class and motherclass. Make sure that all patterns are grouped to one classaccording to criterion of distance or membership grade. Savethe two clustering center data.

Step5. Evaluate number of fault categories in mother andfather class. If it is more than two, go to step 2, else stopfurther clustering. Thus we get a binary tree with K-1 nodes,each node connects two branches. If all fault classes aredivided into a full binary-tree, the lowest level of nodesrepresents each fault class. So the training patterns are dividedinto sub-class groups.

Step6. Construct multi-level of classifiers for identifyingfaulty feature patterns grouped according to the structure offault group binary-tree. Two ways are provided to build multi­level classifier frame: (1) K-1 binary-class classifiers (SVMs

(3)

(2)

(1)

center,the clustering[T T T JTWhere V = VI , V2 , •.. , vm is

N m

i, =IIJL~/d(Xi'Vj)i=1 j=1

iterative solution is got by two prerequisites:N b

I[flijJ Xii=1 (. 1 2 )vj = N b } = , ,... ,mI[flij ]i=1

[1]-1m d(x. V.) b-lflij = I( l' } J (i=1,2, ...,N,j=1,2, ...,m)

k=1 dix., Vk )

1218

or NNs) correspond to a node; (2) assembling multi-class NNswhich can identify more than two classes with relativelysimple structure by adjusting class groups in the tree.

Step7. Use the feature patterns of segmented fault groups totrain each classifier selected in step 6, save correspondingparameters. Input testing feature vectors to trained diagnosissystem, the state of CUT will be got after hierarchicalrecognition of multi-classifiers.

IV. SIMULATION AND TEST

A. Experimental circuit andfault classes

The circuit under test chosen is shown in Fig. 2. Thenominal values of its components are also given in the figure.The tolerance of resistors and capacitors is assumed as 5%.There are fifteen categories of circuit states, includingfourteen faulty conditions and fault free status. Set fault classsymbol as Table I.

*caj_l(k)= L h (1-2k)caj(k)

kEZ

*cd j_1(k)= L g (1-2k)caj (k)kEZ

5000150001.5KO4.5KOlKO3KO2KO6KO2KO6KO2.5nf7.5nf2.5nf7.5nf

Faulty Value

lKOlKO3KO3KO2KO2KO4KO4KO4KO4KO5nf5nf5nf5nf

NominalNFRl~

RliR2~

R2iR3~

R3iR4~

R4iR5~

R5icuerrC2~

C2i

Fault ClassFOFlF2F3F4F5F6F7F8F9

FlOFllF12F13F14

R2

3k +15V

Rl Cl +

+ ~

lk Vo5nF OUT

C2 R3

Vi 5nF 2k R4

4k -15V

R5

4k

Fault ID

Fig.2. Sallen-key bandpass filter

Suppose each component has two fault states which are50% higher or lower than their respective nominal valueslabeled as symbol t or-l-, To generate training data fordifferent fault class, we set a faulty component in the circuitand vary the other resistors and capacitors within theirtolerances. Pspice software is used for circuit modeling andsimulating. A single impulse signal of height 5V and duration1Ous is used as input signal. The output response signals ofthe circuit Vo are sampled for 4-level wavelet transform asdescribed in section III. 50 Monte Carlo analyses areconducted for every fault class of the circuit with tolerance toextract feature vectors by wavelet decomposition. The total750 feature samples are used for fuzzy clustering and trainingclassifiers in each level of diagnosis system. Another 100Monte Carlo analyses are performed to extract testing featurepatterns for fault free and each fault class, which are used fordiagnosis later.

TABLE IFault Classes in Sallen-key Filter Indicated with Nominal and Faulty

Component Values

III. FAULT FEATURE EXTRACTION BY WAVELET

TRANSFORM

In analog circuits, voltage output signals are usuallyadopted for fault feature extraction. As wavelet transform canprovide more precise decomposition in whole frequency bandof an original signal, it has been applied broadly to analyze thecomplicated signals in fault detection fields, because of goodcapability of feature extraction and localization in time andfrequency domain. Various impulse responses under differentstates involve abundant information with various frequencybands, which can characterize different fault status. They aresuitable for being decomposed by wavelet transform as faultfeature vectors.

In wavelet multi-resolution decomposition, output voltagesignal of a CUT can be expressed as:

J(t) = Lcaj(k)lpj,k(t) + Lcdj(k)lf/j,k(t)kEZ kEZ (4)

Where (jJj,k (t) is scaling function and l//j,k (t) is wavelet

function. The approximation signals and the detail signals of

fault signal J(t) can be expressed with approximation

coefficients caj and detail coefficients cdj , which can be

obtained through the low pass filter h* and the high pass

filter g * , the recurrence formula is as follows:

(5)Different level of approximation and detail coefficient

reflect the signal features in low- and high- frequency band.Approximation coefficients represent the basic structure of asignal, while detail coefficients characterize the highfrequency behavior of a signal. There are many ways toconstruct feature vectors with wavelet decompositioncoefficients. In this paper, we choose absolute value sum ofdecomposition coefficients of each level to form N dimensionof feature vectors in N-Ievel transform. Note that it'snecessary to seek a suitable mother wavelet function whichcan generate distinct features across fault classes. Daubechieswavelet is selected by comparing the result of differentwavelets in this paper.

1219

by our approach.TABLE II

Comparision of Reference and Proposed Method

82%

98.9%

98. 5%

Accuracy

100

100

100

No. of Sample

50

50

50

Training TestingMethod

BP-NN[1]

FCM-NN[6]

FCM-SVMs[14]

[*] indicates classifier number used for fault diagnosis in sample circuit.

v. CONCLUSION

In this paper, we propose an approach for analog circuitfault diagnosis based on FCM clustering and multi-classifierfusion. Using detail coefficients of wavelet transform invarious levels as fault feature vectors, FCM clustering isadopted to generate binary tree by pre-classifying the featurevectors into fault groups with small number of fault classes.Hierarchical multi-classifier diagnosis frame is built accordingto the structure of fault tree. Each classifier was trained withcorresponding fault feature patterns separated by clustering.Simulation results for verification showed that the accuracyratio of fault diagnosis based on presented method is muchhigher than traditional methods with less time-consuming. It iseffective for fault diagnosis of analog circuits with largenumber of fault categories.

REFERENCES

[1] S. Yang and S. Tong, Fault diagnosis and testability design in analogsystem, Beijing: Tsinghua University Press,200 1.

[2] M. Aminian and F.Aminian, "A comprehensive Examination of NeuralNetwork Architectures for Analog Fault Diagnosis," IEEE Proceeding,pp.2304-2307,2000.

[3] Aminian, M. and F. Aminian, "Neural-Network Based Analog-CircuitFault Diagnosis Using Wavelet Transform as Processor," IEEE Trans.on Circuits and System II: Analog and Digital signal processing, vol. 47,no.2, pp.15I-I56, 2000.

[4] F. Aminian and M Aminian. "Fault Diagnosis of Analog Circuits UsingBayesian Neural Networks with Wavelet Transform asPreprocessor,"Journal of Electronic Testing: Theory andApplications,voI.I7, no.I, pp. 29-36,2001.

[5] G. Liang, Y. HE, and Y. ZHU, "A method for fault diagnosis of analogcircuits using fuzzy neural network combined with geneticalgorithms,"Journal of Circuits and Systems, vol. 9,no.2, pp.54-57, Feb.2004.

[6] R.Salat and S.Osowski, "Analog filter diagnosis using Support VectorMachine," ECCTD'03,Krakow, III,pp.42I-424, 2003.

[7] Y. Sun, G.Chen, and H. Li . "Support Vector Machine for AnalogCircuit Fault Diagnosis," Journal of Electronic Measurement andInstrument., vol. 22, no.2, pp.72-75, Apr. 2008.

[8] Y. Zhang, X. Wei, and H. Jiang, "Study on the Fault Re-ClassificationMethod in Analog Circuit Fault Diagnosis," Journal of Beijing JiaotongUniversity,voI.30, no.2,pp.53-56,Apr. 2006.

[9] Z. Wei, L. Wang and Y. Li. "Fault diagnosis in Analog Circuits Basedon Dual Neural Networks," Low-voltage Electronic Instruments,no.3,pp.I7-I9, Mar. 2008.

[10] B. Li, M. Shao, and 1. Huang, Principle and application of patternrecognition,.Xi'an: Xidian University Press House,2008.

B. Classifier Selection

As we see in Section II, two types of classifier are properfor our diagnosis system: SVMs or neural networks. SVM is amachine learning method based on structural riskminimization principle with small quantity of samples. As abinary-class classifier, it is designed to separate only twoclasses from each other. To deal with multi-class classification,multiple SVMs are assembled with some strategies, such asone-a-one SVM and DAG-SVM method. They are effective toidentify multi-class, but efficiency of classifying data willdecline as the number of SVM grows superlinearly with theclass number. In this paper, less number of SVM is neededafter FCM clustering. Note that the fault group tree has K-1nodes after binary-class fuzzy clustering, only K-1 SVMs arerequired to separate K fault states completely.

Neural networks have been widely used for fault diagnosisof analog circuits because of their strong capability in tacklingclassification and nonlinear problem in recent years. A multi­class neural network can deal with classification more thantwo classes, hence less number of neural network classifiers ispossibly expected comparing with the method of SVMs in thisdiagnosis system. In this way, it's necessary to calculate thedistance or member grade between testing patterns to the node(clustering centers) of fault binary tree up-to-down. The resultof pre-classification will decide which classifier the testingpatterns will go for.

After knowing the characteristics of neural network andSVM, we can integrate their advantage to construct aclassification system according to the clustering results.

C. Fault Diagnosis

We build two types of multi-classifier frame mentioned asbefore and compare their experimental results with traditionaldiagnosis method based on neural network. Integration ofmulti-NNs is considered firstly because of smaller number ofclassifiers. We obtained fault binary tree with five fault groupsby FCM clustering, which are G1(F1,F3,F7), G2(F6,F10,F13),G3(F4,F11,F12), G4(F8,F5) and G5(F2,F9, F14). Gland G2relate to the first-level father node and the rest three belong tomother node. The mother node generate three groups in whichG4 and G5 lie in the lowest level, and G3 stays at their upperlevel. In our experiments, five BP neural networks are trainedfor identifying five groups of faults. Six BP NN classifiersconstitute this diagnosis system, including a binary-classneural network for distinguish fault free or faulty state. If weadopt the scheme ofmulti-SVMS, the five fault groups shouldbe further divided to generate a full binary tree with 13 two­classification nodes. Consequentially, 14 SVM classifiers arerequired in this system.

The diagnostic results are showed in Table II, which are incontrast with the method based on traditional method [3]. Thetotal correct rate (more than 98%) of our approach is higherthan that of method [3]. As the structure of classifiers in ourapproach is much simpler than that of [3], the cost of trainingand testing time is greatly reduced too. Additionally, the twooverlapped fault classes mentioned in [9] can be well isolated

1220