9
Research Article Study on Support Vector Machine-Based Fault Detection in Tennessee Eastman Process Shen Yin, 1 Xin Gao, 1 Hamid Reza Karimi, 2 and Xiangping Zhu 1 1 College of Engineering, Bohai University, Liaoning 121013, China 2 Department of Engineering, Faculty of Engineering and Science, University of Agder, 4898 Grimstad, Norway Correspondence should be addressed to Xin Gao; [email protected] Received 19 February 2014; Accepted 1 April 2014; Published 24 April 2014 Academic Editor: Hui Zhang Copyright © 2014 Shen Yin et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. is paper investigates the proficiency of support vector machine (SVM) using datasets generated by Tennessee Eastman process simulation for fault detection. Due to its excellent performance in generalization, the classification performance of SVM is satisfactory. SVM algorithm combined with kernel function has the nonlinear attribute and can better handle the case where samples and attributes are massive. In addition, with forehand optimizing the parameters using the cross-validation technique, SVM can produce high accuracy in fault detection. erefore, there is no need to deal with original data or refer to other algorithms, making the classification problem simple to handle. In order to further illustrate the efficiency, an industrial benchmark of Tennessee Eastman (TE) process is utilized with the SVM algorithm and PLS algorithm, respectively. By comparing the indices of detection performance, the SVM technique shows superior fault detection ability to the PLS algorithm. 1. Introduction Fault detection in manufacture process aims at timely nosing out abnormal process behavior. Early fault detection of process makes the process safer, more efficient and more economical and guarantees the quality of products. ere- fore, several statistical methods appeared for fault diagnosis and detection, such as artificial neural networks (ANN), fuzzy logic systems, genetic algorithms, principal component analysis (PCA), and more recently support vector machine (SVM) [1]. SVM has been extensively studied and has been used to construct data-driven modeling, classifying, and fault detection due to its better generalization ability and nonlinear classification ability [24]. On the other hand, since massive amounts of data produced in the industrial manufacturing process can be recorded and collected, the study of data-driven technique has become an active research field. It can benefit diverse communities including process engineering [5, 6]. Over the past few years, linear supervised classification technique, for example, K-nearest neighbor [7], PCA [8, 9], Fisher discriminant analysis [8], discriminant partial least squares (DPLS) [810], and nonlinear classi- fication technique, for example, ANN [11] and SVM [12], have been proposed and greatly improved [13, 14]. Moreover, several machine learning algorithms have been applied in real processes and simulation processes [1517]; for example, the PCA algorithm is used to analyse product quality for a pilot plant [18], PLS and PCA are applied in chemical industry for process fault monitoring [19], and the key performance indicator prediction scheme is applied into an industrial hot strip mill in [20]. e SVM algorithm can improve the detection accuracy and it started to be used for fault detection. Now it has been extensively used to solve classification problems in many domains, for example, face, object and text detection and categorization, information and image retrieval, and so forth. e algorithm of support vector machine (SVM) is studied detailedly in this paper. SVM is a representative nonlinear technique and it is a potentially effective technique for classifying all kinds of datasets [21]. Fault detection can be considered as a special classification problem involved in model-based method [22] and data-based method, with the purpose to timely recognise faulty condition. With the help of cross-validation algorithm to optimise parameters, the per- formance of classification is greatly enhanced [2325]. en, to test the classification performance of SVM algorithm, Hindawi Publishing Corporation Abstract and Applied Analysis Volume 2014, Article ID 836895, 8 pages http://dx.doi.org/10.1155/2014/836895

Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

Embed Size (px)

Citation preview

Page 1: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

Research ArticleStudy on Support Vector Machine-Based Fault Detection inTennessee Eastman Process

Shen Yin1 Xin Gao1 Hamid Reza Karimi2 and Xiangping Zhu1

1 College of Engineering Bohai University Liaoning 121013 China2Department of Engineering Faculty of Engineering and Science University of Agder 4898 Grimstad Norway

Correspondence should be addressed to Xin Gao gaoxinjessicagmailcom

Received 19 February 2014 Accepted 1 April 2014 Published 24 April 2014

Academic Editor Hui Zhang

Copyright copy 2014 Shen Yin et alThis is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

This paper investigates the proficiency of support vector machine (SVM) using datasets generated by Tennessee Eastman processsimulation for fault detection Due to its excellent performance in generalization the classification performance of SVM issatisfactory SVM algorithm combined with kernel function has the nonlinear attribute and can better handle the case wheresamples and attributes are massive In addition with forehand optimizing the parameters using the cross-validation techniqueSVM can produce high accuracy in fault detectionTherefore there is no need to deal with original data or refer to other algorithmsmaking the classification problem simple to handle In order to further illustrate the efficiency an industrial benchmark of TennesseeEastman (TE) process is utilized with the SVM algorithm and PLS algorithm respectively By comparing the indices of detectionperformance the SVM technique shows superior fault detection ability to the PLS algorithm

1 Introduction

Fault detection in manufacture process aims at timely nosingout abnormal process behavior Early fault detection ofprocess makes the process safer more efficient and moreeconomical and guarantees the quality of products There-fore several statistical methods appeared for fault diagnosisand detection such as artificial neural networks (ANN)fuzzy logic systems genetic algorithms principal componentanalysis (PCA) and more recently support vector machine(SVM) [1] SVM has been extensively studied and has beenused to construct data-driven modeling classifying andfault detection due to its better generalization ability andnonlinear classification ability [2ndash4] On the other handsince massive amounts of data produced in the industrialmanufacturing process can be recorded and collected thestudy of data-driven technique has become an active researchfield It can benefit diverse communities including processengineering [5 6] Over the past few years linear supervisedclassification technique for example K-nearest neighbor [7]PCA [8 9] Fisher discriminant analysis [8] discriminantpartial least squares (DPLS) [8ndash10] and nonlinear classi-fication technique for example ANN [11] and SVM [12]

have been proposed and greatly improved [13 14] Moreoverseveral machine learning algorithms have been applied inreal processes and simulation processes [15ndash17] for examplethe PCA algorithm is used to analyse product quality fora pilot plant [18] PLS and PCA are applied in chemicalindustry for process fault monitoring [19] and the keyperformance indicator prediction scheme is applied into anindustrial hot strip mill in [20] The SVM algorithm canimprove the detection accuracy and it started to be used forfault detection Now it has been extensively used to solveclassification problems in many domains for example faceobject and text detection and categorization information andimage retrieval and so forth

The algorithm of support vector machine (SVM) isstudied detailedly in this paper SVM is a representativenonlinear technique and it is a potentially effective techniquefor classifying all kinds of datasets [21] Fault detection canbe considered as a special classification problem involved inmodel-based method [22] and data-based method with thepurpose to timely recognise faulty conditionWith the help ofcross-validation algorithm to optimise parameters the per-formance of classification is greatly enhanced [23ndash25] Thento test the classification performance of SVM algorithm

Hindawi Publishing CorporationAbstract and Applied AnalysisVolume 2014 Article ID 836895 8 pageshttpdxdoiorg1011552014836895

2 Abstract and Applied Analysis

a simulation model the Tennessee Eastman process is usedfor detecting fault which has 52 variables representing thedynamics of the process [26] In this simulation originaldataset is handled using SVM algorithm and it obtainssatisfactory fault detection result In the process of modelbuilding and test data classifying no other theory is addedrelatively decreasing the calculation time and reducing thecomputational burden Compared with PLS algorithm clas-sifier based on SVM performs higher accuracy Finally usingthe SVM-based classifier with optimal parameters faultystation of the process is detected

The paper is arranged as follows The SVM classificationalgorithm PLS algorithm and cross-validation algorithm areintroduced in the next section Sections 3 and 4 present anapplication to Tennessee Eastman process simulator usingSVM and PLS algorithms respectively and SVM-basedfault detection outperforms that of PLS algorithm Section 5summarizes a conclusion

2 Method Algorithms

21 Support VectorMachinesTheory Support vectormachine(SVM) is a relatively new multivariate statistical approachand has become popular due to its preferable effect ofclassification and regression SVM-based classifier has bettergeneralization property because it is based on the structuralrisk minimization principle [27] SVM algorithm has thenonlinear attribute thus it can deal with large feature spaces[28] Due to the aforementioned two factors SVM algorithmbegins to be used in machine fault detectionThe fundamen-tal principle of SVM is separating dataset into two classesaccording to the hyperplane (a decision boundary) whichshould have maximum distance between support vectors ineach class Support vectors are representative data pointsand their increasing number may increase the complexity ofproblem [28 29]

This thesis uses a binary classifier with dataset and thecorresponding labels Training dataset containing two classesis given in matrix 119883 with the form of 119898 times 119899 [6] in which119898 represents the number of observe samples while 119899 standsfor the quantity of the observed variables 119909

119894is denoted as a

column vector to stand for the 119894th row of 119883 Each sample isassumed to be in a positive class or in a negative class Besidesa column vector 119884 serves as the class label containing twoentries minus1 and 1 Denote that 119910

119894= 1 is associated with one

class and 119910119894= minus1 with the other class If the training dataset

is linearly separable the SVMwill try to separate it by a linearhyperplane

119891 (119909) = ⟨119908 119909⟩ + 119887 = 0 (1)

where 119908 is an119898-dimensional vector and 119887 is a scalarThe parameters 119908 and 119887 decide the separating hyper-

planersquos position and orientation A separating hyperplane isconsidered to be optimal if it creates maximum distancebetween the closest vectors and the hyperplane The closestpoints in each class are denoted as support vectors If otherpoints in the training set are removed the calculated decisionboundary remains the same one That is to say the support

vectors contain all information in the dataset to define thehyperplane The distance 119889 from a data point 119909

119894to the

separating hyperplane is

119889 =|⟨119908 119909⟩ + 119887|

119908 (2)

Vapink in 1995 put forward a canonical hyperplane [30]where 119908 and 119887 should satisfy

min119894

1003816100381610038161003816⟨119908 119909119894⟩ + 1198871003816100381610038161003816 = 1 (3)

That is to say if the nearest point is taken to the hyperplanefunction the result is constrained to be 1 This restriction onthe parameters is to simplify the formation of problem In away as for a training data 119909

119894 119910119894 the separating hyperplane of

the above-mentioned form will become

119910119894119891 (119909119894) = 119910119894(⟨119908 119909

119894⟩ + 119887) ge 1 119894 = 1 119898 (4)

The best separating hyperplane is the one that makesmaximum distance from support vectors to decision bound-ary The maximum distance is denoted as 120588

Consider

120588 =1

119908 (5)

Hence as for linear separable data the optimal separatinghyperplane satisfies the following function

min 120601 (119908) =1

21199082 (6)

To solve the optimal problem equation (6) under the con-strain of (4) define the Lagrangian to be

ℓ (119908 119887 120572) =1

21199082minus

119898

sum

119894=1

120572119894[119910119894(⟨119908 119909

119894⟩ + 119887) minus 1] (7)

where 120572119894is called Lagrangian multiplier The Lagrangian

should be maximised by choosing appropriate 120572119894and should

be minimised by 119908 119887Taking the noise in the data and the misclassification of

hyperplane into consideration the above function describingthe separate hyperplane equation (4) is not accurate enoughTo make the optimal separating boundary to be generalisedwe reformulate the described function of the separate hyper-plane

119910119894(⟨119908 119909

119894⟩ + 119887) ge 1 minus 120585

119894 119894 = 119894 119898 (8)

where the variable 120585119894represents a measure of distance from

hyperplane to misclassified points and 120585119894

ge 0 To findthe optimal generalised separating hyperplane the followingoptimal problem should be solved

min119908119887

1

21199082+ 119862

119898

sum

119896=1

120585119894

st 119910119894(⟨119908 119909

119894⟩ + 119887) ge 1 minus 120585

119894

120585119894ge 0 119894 = 119894 119898

(9)

Abstract and Applied Analysis 3

where the parameter 119862 a given value is called error penaltyAs for the above-mentioned data inseparable case in order tosimplify the optimal problem define the Lagrangian to be

ℓ (119908 119887 120585 120572 120574) =1

21199082+ 119862

119898

sum

119894=1

120585119894

minus

119898

sum

119894=1

120572119894[119910119894(⟨119908 119909

119894⟩ + 119887) minus 1 + 120585

119894] minus

119898

sum

119894=1

120574119894120585119894

(10)

where 120572 120574 are the Lagrangian multipliers We consider theminimization problem as original primal problem

Consider

min119908119887120585

120579119901 (119908) = min

119908119887120585

max120572120574

ℓ (119908 119887 120585 120572 120574) (11)

When satisfying the Kuhn-Tucker condition then theprimal problem is transformed to its dual problem which is

max120572120574

120579119889(120572 120574) = max

120572120574min119908119887120585

ℓ (119908 119887 120585 120572 120574) (12)

Then the task is minimizing ℓ in (10) by adjusting thevalue of 119908 119887 120585 At the optimal point derivatives of ℓ shouldbe zero The saddle-point equation is as follows

120597ℓ

120597119908= 0 997904rArr

119898

sum

119894=1

120572119894119910119894= 0 (13)

120597ℓ

120597119887= 0 997904rArr 119908 =

119898

sum

119894=1

120572119894119910119894119909119894 (14)

120597ℓ

120597120585= 0 997904rArr 120572

119894+ 120574119894= 119862 (15)

If we take (13) (14) and (15) back into (10) we can obtainthe dual quadratic optimization problem [30 31]

max120572

119882(120572) = max120572

minus1

2

119898

sum

119894=1

119898

sum

119895=1

120572119894120572119895119910119894119910119895⟨119909119894 119909119895⟩ +

119898

sum

119896=1

120572119896

(16)

Satisfying the constrains

0 le 120572119894le 119862 119894 = 1 119898

119898

sum

119894=1

120572119894119910119894= 0

(17)

When solved the dual quadratic optimization problemshown in (16) the 120572

119894will be obtained Then take a look back

at (14) and describe the optimal 119908 using 120572119894and the form of

hyperplane can be changed to

⟨119908 119909119894⟩ + 119887 =

119898

sum

119894119895=1

120572119894119910119894⟨119909119894 119909119895⟩ + 119887 (18)

Table 1 Formulation of Kernel functions

Kernel 119870(119909119894 119909119895)

Linear 119909119879

119894sdot 119909119895

Polynomial (120574119909119879119894sdot 119909119895+ constant)119889

Gaussian RBF functions 119890minus120574||119909119894minus119909119895 ||

2

120574 constant and 119889 are kernel parameters

The classifier implementing the optimal separating hyper-plane comes out in the following form

119891 (119909) = sgn(119898

sum

119894119895=1

120572119894119910119894⟨119909119894 119909119895⟩ + 119887) (19)

However in some cases linear classifier is not suitable forexample data is overlapped or cannot be linearly separatedTherefore the input vectors should be projected into a higherdimensional feature space and there the data may be linearlyclassified more efficiently with the use of SVM algorithmHowever it may cause computational problem due to thelarge vectors and high dimensionality The idea of usingKernel function enables the calculation performed in theoriginal space instead of in the projected high dimensionedfuture space avoiding the curse of dimensionality [27 31]Given a feature mapping 120601 we define the correspondingKernel function to be

119870(119909119894 119909119895) = ⟨120601 (119909

119894) 120601 (119909

119895)⟩ (20)

Thus the linear decision hyperplane in the high featurespace is

119891 (119909) = sgn(119898

sum

119894119895=1

120572119894119910119894⟨119870 (119909

119894 119909119895)⟩ + 119887) (21)

119870(119909119894 119909119895) is inexpensive to calculate Kernel function returns

an inner production of vectors in the high feature spacewhichis evaluated in the original space not in high feature spaceThe commonly used four Kernel functions are as shown inTable 1

22 SVM Model Selection SVM algorithm is a very effectivedata classifying technique and building a model to detectfault based on SVM algorithm is not so complex Trainingdataset and testing sets are usually involved when a classi-fication is done Besides every instance in the training setcontains two parts one is target value (ie the class label)and the other is several attributes (ie observed variables)The basic process of using SVM algorithm to classify data isas follows at first build a classifier based on the training setand then use it to predict the target value of the data in testingset where only the attributes are known

To construct a classifier model and figure out faulty datathe following procedure is used

(i) Transform data collected from real process to theformat that SVM classifier can use

4 Abstract and Applied Analysis

(ii) Try a few kinds of Kernels and find out the best onethen search the optimal parameters for it This thesisuses Gaussian RBF Kernel function and the optimalparameters are obtained by using the cross-validationalgorithm

(iii) Use the optimal parameters and appropriate Kernelfunction to build a classifier

(iv) Take testing data into the constructed classifier anddo a test As a result the faulty data will be figured outand in this way the faulty station can be detected

221 RBF Kernel Which kind of Kernel functions to useis significant because it decides the feature space wheretraining data is classifiedTheoretically the Kernel function isconsidered to be better if it can provide higher upper boundon the VC dimension However it requires evaluating thehypersphere enclosing all data in the nonlinear feature spacewhich is difficult to accomplish Among the commonly usedKernel functions for the SVM algorithm the RBF Kernel ispreferable according to practical experience

Kernel function can nonlinearly project original inputdata to a higher dimensional space thus SVM equippedwith Kernel function is able to deal with the case where theinput data cannot be linearly separated Generally speakingthe RBF Kernel is the first choice for the following tworeasons Firstly linear Kernel is a special RBF Kernelmdashwhenthe parameter 119862 is adjusted to a certain value linear Kernelbehaves similarly to RBF Kernel and so it is the sigmoidKernel at some parameters Secondly the RBF Kernel bringsfewer computational costs and fewer hyperparameters [23]

222 Cross-Validation When using an RBF Kernel func-tion it needs appropriate parameters to make sure that theclassifier accurately predicts unknown data It is not knownbeforehand what the best parameter values are neverthelessthe optimal parameter searching can be accomplished usingcross-validation algorithm [23]

The cross-validation algorithm is a model validationmethod which is used to evaluate the accuracy of a predictivemodel The goal of cross-validation is to give an insight onhow the model generalizes to an independent dataset (ie anunknown dataset) by defining a dataset in the training phaseto test the model

In the process of 119896-fold cross-validation original trainingset is randomly divided into k equal size parts Sequentiallyone subset is used as the testing dataset to test the predictivemodel and the rest of 119896 minus 1 subsets are combined asthe training dataset The aforementioned validation processshould be repeated 119896 times in all with every subset performedas testing data once Using the 119896 results the predictive resultof themodel is produced and it is the overall misclassificationrate across all testing sets

This thesis uses 5-fold cross-validation algorithm to findout the overallmisclassification rate across all testing setsThecross-validation process should be performed many times topick out the parameters whichmake the overall misclassifica-tion minimise In this way the optimal parameters are foundand the classifier can obtain the best accuracy rate

23 PLS for Fault Detection ThePLS algorithm is introducedby Dayal and Macgregor [32] due to its simplicity and thelesser computational effort when dealing with the processmonitoring with large data In this technique we also denotetraining data in the form of 119898 times 119899 matrix 119883 in which 119898

represents the number of observed samples while 119899 is thequantity of the attributes and 119884 isin R119898times1 The PLS algorithmprojects thematrix119883 into a lowdimensional spacewith latentvariables and 119884 can be constructed by these latent variablesThe construct model is as follows

119883 = 119879119875119879+ 119864 =

119860

sum

119894=1

119905119894119901119879

119894+ 119864

119884 = 119879119876119879+ 119865 =

119860

sum

119894=1

119905119894119902119879

119894+ 119865

(22)

where119879 = [1199051 119905119860] is the scorematrix of119883119875 = [119901

1 119901119860]

and 119876 = [1199021 119902119860] are the loading matrix of 119883 and 119884

respectively The latent variables 119905119894isin 119879 (119894 = 1 119860) can be

directly calculated from 119883 by 119905119894= 119883119903119894 119877 = [119903

1 119903119860] and 119903

119894

are calculated by the following

1199031= 1199081

119903119894= 119908119894minus 119901119879

11199081198941199031minus 119901119879

119894minus1119908119894119903119894minus1

119894 = 2 119860

(23)

where the 119908119894(119894 = 1 119860) is the weight vector of the 119894th

deflated119883 PLS decomposes119883 into two parts

119909 = 119909 + 119909

119909 = 119875119877119879119909 isin 119878119909

119909 = (119868 minus 119875119877119879) 119909 isin 119878

119909

(24)

Usually1198792 statistic is used to detect abnormalities and thecalculation method is

1198792= 119909119879119877(

119879119879119879

119899 minus 1)

minus1

119877119879119909 (25)

With a given confidence level 120572 the threshold for 1198792 willbe calculated by the following

119869th1198792 =119860 (1198992minus 1)

119899 (119899 minus 119860)119865119897119899minus119897120572

(26)

where 119865119897119899minus119897120572

represents 119865-distribution with l and 119873 minus 1

degrees of freedom and its confidence level is 120572 If 1198792 are allless than their corresponding thresholds the process is out offault [32]

3 Simulation

31 Tennessee Eastman Process The Tennessee Eastmanprocess (TE process) is a simulation model of a practicalindustrial process The simulation model is very suitable forresearching onprocess control technology and also applicable

Abstract and Applied Analysis 5

A

D

E

C

XCXD

XEXF

FI

FI

FI

LI

LI

LISC

Ana

lyze

r

Ana

lyze

rA

naly

zer

PI

PI

PI

FI

FI

JI

TITI

TI

TI

TIFI

FI

FI

FI

FI

XAXB

XD

XE

XF

XG

XAXB

XH

XCXD

XEXFXGXH

Vapliq separator

Compressor

Purge

Product

CWS

CWS

CWR

CWR

Reactor

Stripper

Cond

Stm

Condenser

4

6

3

2

1

7

12

5

89

10

11

Figure 1 The flowsheet of TE process

for other kinds of control issues Figure 1 shows the flowsheetof TE process The TE process consists of five major partsthe reactor the product condenser a vapor-liquid separator arecycle compressor and a product stripper to help accomplishreactor separator and recycle arrangement More details canbe found in [33 34]

32 Simulated Data Intro The TE process is a plant-wideclosed-loop control structure The simulated process canproduce normal operating condition as well as 21 faulty con-ditions and generate simulated data at a sampling interval of3min For each case (no matter normal or faulty condition)two sets of data are produced training datasets and testingdatasets The training sets are used to construct statisticalpredictive model and the testing datasets are used to estimatethe accuracy of the built classifier In training sets the normaldataset contains 500 observation samples while each faultydataset contains 480 observation samples As for testing dataeach dataset (both normal and faulty conditions) consists of960 observations In the faulty condition faulty informationemerges 8h later since the TE process is turned on Thatis to say in each faulty condition the former 160 samplesare shown normally while the remaining 800 instances areshown really faulty and should be detected Every observationsample contains 52 variables which consists of 22 processmeasure variables 19 component measure variables and 12control variables All the datasets used in this paper can befound in [1]

In the following section the fault detection result of SVM-based classifier will be compared with the one based on

the PLS algorithm using the dataset generated by TE processsimulator For SVM classifier the RBF Kernel is used and therelated parameters will be set ahead according to the result ofcross-validation to make sure of the classifierrsquos high accuracyIn the comparison the performance is measured by faultdetection rates

4 Result and Discussion

TE process simulator can generate 22 types of conditionscontaining the normal condition and 21 kinds of pro-grammed faults caused by various known disturbances inthe process Once fault is added all variables will be affectedand some changes will emerge According to Chiang et al[8] and Zhang [18] detection for faults 3 9 15 and 21 isvery difficult for there are not any observable changes in themeans variance or the peak timeTherefore these four faultsalways cannot be detected by any statistics technique thusthe four faults are not analysed in this paperThe informationof all the faults is presented in Table 2

In order to profoundly test the detection effect of theclassifier based on theory of SVM the SVM algorithm andPLS algorithm are applied to the TE process respectively Asfor SVM classifier all faults data are by turns combined withthe normal condition data as the dataset for binary classifierAfter models are built by training data we use testing data toevaluate the prediction result through the following commonindices accuracy (Acc) and fault detection rate (FDR) [35]Then the detection result for each fault is shown in Table 3Table 4 represents the detection result using PLS technique

6 Abstract and Applied Analysis

Table 2 Descriptions of 21 faults in TE process

Number Fault descriptionIDV0 NormalIDV1 119860119862 feed ratio 119861 composition constantIDV2 B composition 119860119862 ratio constantIDV4 Reactor cooling water inlet temperatureIDV5 Condenser cooling water inlet temperatureIDV6 A feed loss (stream 1)IDV7 C header pressure loss-reduced availability (stream 4)IDV8 A B and C feed composition (stream 4)IDV10 C feed temperature (stream 4)IDV11 Reactor cooling water inlet temperatureIDV12 Condenser cooling water inlet temperatureIDV13 Reaction kineticsIDV14 Reactor cooling water valveIDV16 UnknownIDV17 UnknownIDV18 UnknownIDV19 UnknownIDV20 Unknown

Table 3 Results of faults detection with one against one classifier

Fault Acc () FDR ()IDV1 9844 995IDV2 9812 9812IDV4 999 9988IDV5 9198 9075IDV6 6677 6013IDV7 9958 9891IDV8 9635 96IDV10 7771 81IDV11 7302 8025IDV12 974 9775IDV13 9333 925IDV14 8906 91IDV16 8073 8938IDV17 8385 8163IDV18 901 895IDV19 7448 8588IDV20 775 805

Consider

(i) Acc =the quantity of data correctly predicated

the quantity of all data

times 100

(ii) FDR =number of faulty data correctly predicated

number of all faulty data

times 100

(27)

Table 4 Results of faults detection using PLS algorithm

Fault FDR ()1198792

1199091198792

119909

IDV1 9937 9962IDV2 9875 9812IDV4 063 9725IDV5 1665 3004IDV6 99 100IDV7 2916 100IDV8 8986 9687IDV10 1815 4355IDV11 263 6884IDV12 7685 9837IDV13 9262 9499IDV14 225 100IDV16 939 2941IDV17 488 9174IDV18 8748 8986IDV19 038 1514IDV20 184 5044

Figure 2 shows the predicted label119910 using three classifierswith the training and testing data respectively come fromnormal condition and fault 1 normal condition and fault2 and normal condition and fault 4 The first 160th testthe process shows normally and 119910 should be minus1 The faultinformation appears at the moment of 161st sample-takenand the label y should be 1 since then From Figure 2 wecan see that the SVM classifierrsquos prediction result for mostof the time is right In addition Table 3 presents detaileddetection indices of these 21 SVM classifiers It is worthmentioning that the hyperparameters of the classifiers are alloptimized beforehand Without the optimization of classifierparameter the predicted effect will not be so goodMoreoverthe index accuracy will be 1677 when using classifierrsquosdefault parameters to detect fault 1 Table 4 shows the detailedindices using PLS technique to detect faults It can be seenthat the SVM classifiers with optimal hyperparameters aremostly able to detect the faulty data and the accuracy ishigher than that given by using PLS algorithmTherefore theSVM algorithm has a better performance of fault detection

To further test the predictive ability of the SVM classifierthat is the detective ability of fault in the TE process we usenormal condition data combined with three faulty conditiondatasets (fault 1 fault 2 fault 4) as the training data toconstruct a classificationmodel and then used correspondingtest data as testing data to observe the classification result Asshown in Table 5 the detection indices are also good thoughthe computing time is a little longer with the same computefacility Thus the SVM algorithm can perform satisfactorilyon the original dataset containing 52 attributes without anytransformation In this way in real process with advancedcompute facility we can train normal data and all faults datato build a classifier Once the fault label is figured out the faultin process is detected

Abstract and Applied Analysis 7

0 200 400 600 800 1000minus2

0

2

y(0

vers

us4

)SVM(one versus one) on test data

(a)

y (0

ver

sus 2

)

0 200 400 600 800 1000minus2

0

2SVM(one versus one) on test data

(b)

0 200 400 600 800 1000minus2

0

2y

(0ve

rsus

4)

SVM(one versus one) on test data

(c)

Figure 2 Classification results for the testing dataset by the SVM-based classifier with optimal parametersThe value 119910 = minus1 represents at thesample-taken moment that the process is in normal condition while 119910 = 1 stands for faulty conditionThe top plot is the classification resultbetween normal condition and fault 1 the middle one is the result between normal condition and fault 2 and the last plot is result betweennormal condition and fault 4 In the simulation of fault condition the fault information appears from the moment of 161th sample-taken andit is pointed out by red line

Table 5 Results of faults detection with one against all classifier

Test description Test data Acc () FDR ()0 versus 1 2 IDV1 test data 9802 99620 versus 1 2 IDV2 test data 9583 9850 versus 1 2 IDV0 test data 83440 versus 1 2 4 IDV1 test data 9646 99750 versus 1 2 4 IDV2 test data 95 98880 versus 1 2 4 IDV4 test data 9698 1000 versus 1 2 4 IDV0 test data 7962

From the above-shown result of fault detection on TEprocess we can conclude that the classifier based on SVMalgorithm is of good predictive ability In addition there aretwo facts that should be mentioned First before detectingnormal condition or faulty condition we have used thetechnique of cross-validation to optimize classifierrsquos hyperpa-rametersTherefore the performance of classifier could be thebest Second the classification on TE process based on SVMalgorithm performs satisfactorily without any other processfor example foregoing data dealing process and attributesselection And this featuremakes the SVMclassifier easy to bebuilt Besides the calculation and calculate time is relativelysmall since the algorithms used are fewer

5 Conclusion

TE process a benchmark chemical engineering model isused in this paper for fault detection It can be found thatthe fault detection ability of the classifier based on SVM algo-rithm using the TE processrsquos original data is satisfactory andthis indicates the advantage of using nonlinear classification

when the number of samples or attributes is very large Bycomparing detection performancewith classifier based on thePLS algorithm the classifier based on SVM algorithm withKernel function shows superior accuracy rate In additionparameter optimization beforehand plays a great role inimproving the effectiveness of classification It also simplifiesthe problem and by using no other technique relativelydecreases the computational load to reach a satisfactoryclassification result as well

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

References

[1] S Yin Data-driven design of fault diagnosis systems [PhDdissertation] University of Duisburg-Essen 2012

[2] V N Vapnik The Nature of Statistical Learning TheorySpringer New York NY USA 1995

[3] B E Boser I M Guyon and V N Vapnik ldquoTraining algorithmfor optimal margin classifiersrdquo in Proceedings of the 5h AnnualACM Workshop on Computational Learning Theory pp 144ndash152 New York NY USA July 1992

[4] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[5] A Kulkarni V K Jayaraman and B D Kulkarni ldquoSupportvector classification with parameter tuning assisted by agent-based techniquerdquo Computers and Chemical Engineering vol 28no 3 pp 311ndash318 2004

[6] L H Chiang M E Kotanchek and A K Kordon ldquoFaultdiagnosis based on Fisher discriminant analysis and support

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 2: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

2 Abstract and Applied Analysis

a simulation model the Tennessee Eastman process is usedfor detecting fault which has 52 variables representing thedynamics of the process [26] In this simulation originaldataset is handled using SVM algorithm and it obtainssatisfactory fault detection result In the process of modelbuilding and test data classifying no other theory is addedrelatively decreasing the calculation time and reducing thecomputational burden Compared with PLS algorithm clas-sifier based on SVM performs higher accuracy Finally usingthe SVM-based classifier with optimal parameters faultystation of the process is detected

The paper is arranged as follows The SVM classificationalgorithm PLS algorithm and cross-validation algorithm areintroduced in the next section Sections 3 and 4 present anapplication to Tennessee Eastman process simulator usingSVM and PLS algorithms respectively and SVM-basedfault detection outperforms that of PLS algorithm Section 5summarizes a conclusion

2 Method Algorithms

21 Support VectorMachinesTheory Support vectormachine(SVM) is a relatively new multivariate statistical approachand has become popular due to its preferable effect ofclassification and regression SVM-based classifier has bettergeneralization property because it is based on the structuralrisk minimization principle [27] SVM algorithm has thenonlinear attribute thus it can deal with large feature spaces[28] Due to the aforementioned two factors SVM algorithmbegins to be used in machine fault detectionThe fundamen-tal principle of SVM is separating dataset into two classesaccording to the hyperplane (a decision boundary) whichshould have maximum distance between support vectors ineach class Support vectors are representative data pointsand their increasing number may increase the complexity ofproblem [28 29]

This thesis uses a binary classifier with dataset and thecorresponding labels Training dataset containing two classesis given in matrix 119883 with the form of 119898 times 119899 [6] in which119898 represents the number of observe samples while 119899 standsfor the quantity of the observed variables 119909

119894is denoted as a

column vector to stand for the 119894th row of 119883 Each sample isassumed to be in a positive class or in a negative class Besidesa column vector 119884 serves as the class label containing twoentries minus1 and 1 Denote that 119910

119894= 1 is associated with one

class and 119910119894= minus1 with the other class If the training dataset

is linearly separable the SVMwill try to separate it by a linearhyperplane

119891 (119909) = ⟨119908 119909⟩ + 119887 = 0 (1)

where 119908 is an119898-dimensional vector and 119887 is a scalarThe parameters 119908 and 119887 decide the separating hyper-

planersquos position and orientation A separating hyperplane isconsidered to be optimal if it creates maximum distancebetween the closest vectors and the hyperplane The closestpoints in each class are denoted as support vectors If otherpoints in the training set are removed the calculated decisionboundary remains the same one That is to say the support

vectors contain all information in the dataset to define thehyperplane The distance 119889 from a data point 119909

119894to the

separating hyperplane is

119889 =|⟨119908 119909⟩ + 119887|

119908 (2)

Vapink in 1995 put forward a canonical hyperplane [30]where 119908 and 119887 should satisfy

min119894

1003816100381610038161003816⟨119908 119909119894⟩ + 1198871003816100381610038161003816 = 1 (3)

That is to say if the nearest point is taken to the hyperplanefunction the result is constrained to be 1 This restriction onthe parameters is to simplify the formation of problem In away as for a training data 119909

119894 119910119894 the separating hyperplane of

the above-mentioned form will become

119910119894119891 (119909119894) = 119910119894(⟨119908 119909

119894⟩ + 119887) ge 1 119894 = 1 119898 (4)

The best separating hyperplane is the one that makesmaximum distance from support vectors to decision bound-ary The maximum distance is denoted as 120588

Consider

120588 =1

119908 (5)

Hence as for linear separable data the optimal separatinghyperplane satisfies the following function

min 120601 (119908) =1

21199082 (6)

To solve the optimal problem equation (6) under the con-strain of (4) define the Lagrangian to be

ℓ (119908 119887 120572) =1

21199082minus

119898

sum

119894=1

120572119894[119910119894(⟨119908 119909

119894⟩ + 119887) minus 1] (7)

where 120572119894is called Lagrangian multiplier The Lagrangian

should be maximised by choosing appropriate 120572119894and should

be minimised by 119908 119887Taking the noise in the data and the misclassification of

hyperplane into consideration the above function describingthe separate hyperplane equation (4) is not accurate enoughTo make the optimal separating boundary to be generalisedwe reformulate the described function of the separate hyper-plane

119910119894(⟨119908 119909

119894⟩ + 119887) ge 1 minus 120585

119894 119894 = 119894 119898 (8)

where the variable 120585119894represents a measure of distance from

hyperplane to misclassified points and 120585119894

ge 0 To findthe optimal generalised separating hyperplane the followingoptimal problem should be solved

min119908119887

1

21199082+ 119862

119898

sum

119896=1

120585119894

st 119910119894(⟨119908 119909

119894⟩ + 119887) ge 1 minus 120585

119894

120585119894ge 0 119894 = 119894 119898

(9)

Abstract and Applied Analysis 3

where the parameter 119862 a given value is called error penaltyAs for the above-mentioned data inseparable case in order tosimplify the optimal problem define the Lagrangian to be

ℓ (119908 119887 120585 120572 120574) =1

21199082+ 119862

119898

sum

119894=1

120585119894

minus

119898

sum

119894=1

120572119894[119910119894(⟨119908 119909

119894⟩ + 119887) minus 1 + 120585

119894] minus

119898

sum

119894=1

120574119894120585119894

(10)

where 120572 120574 are the Lagrangian multipliers We consider theminimization problem as original primal problem

Consider

min119908119887120585

120579119901 (119908) = min

119908119887120585

max120572120574

ℓ (119908 119887 120585 120572 120574) (11)

When satisfying the Kuhn-Tucker condition then theprimal problem is transformed to its dual problem which is

max120572120574

120579119889(120572 120574) = max

120572120574min119908119887120585

ℓ (119908 119887 120585 120572 120574) (12)

Then the task is minimizing ℓ in (10) by adjusting thevalue of 119908 119887 120585 At the optimal point derivatives of ℓ shouldbe zero The saddle-point equation is as follows

120597ℓ

120597119908= 0 997904rArr

119898

sum

119894=1

120572119894119910119894= 0 (13)

120597ℓ

120597119887= 0 997904rArr 119908 =

119898

sum

119894=1

120572119894119910119894119909119894 (14)

120597ℓ

120597120585= 0 997904rArr 120572

119894+ 120574119894= 119862 (15)

If we take (13) (14) and (15) back into (10) we can obtainthe dual quadratic optimization problem [30 31]

max120572

119882(120572) = max120572

minus1

2

119898

sum

119894=1

119898

sum

119895=1

120572119894120572119895119910119894119910119895⟨119909119894 119909119895⟩ +

119898

sum

119896=1

120572119896

(16)

Satisfying the constrains

0 le 120572119894le 119862 119894 = 1 119898

119898

sum

119894=1

120572119894119910119894= 0

(17)

When solved the dual quadratic optimization problemshown in (16) the 120572

119894will be obtained Then take a look back

at (14) and describe the optimal 119908 using 120572119894and the form of

hyperplane can be changed to

⟨119908 119909119894⟩ + 119887 =

119898

sum

119894119895=1

120572119894119910119894⟨119909119894 119909119895⟩ + 119887 (18)

Table 1 Formulation of Kernel functions

Kernel 119870(119909119894 119909119895)

Linear 119909119879

119894sdot 119909119895

Polynomial (120574119909119879119894sdot 119909119895+ constant)119889

Gaussian RBF functions 119890minus120574||119909119894minus119909119895 ||

2

120574 constant and 119889 are kernel parameters

The classifier implementing the optimal separating hyper-plane comes out in the following form

119891 (119909) = sgn(119898

sum

119894119895=1

120572119894119910119894⟨119909119894 119909119895⟩ + 119887) (19)

However in some cases linear classifier is not suitable forexample data is overlapped or cannot be linearly separatedTherefore the input vectors should be projected into a higherdimensional feature space and there the data may be linearlyclassified more efficiently with the use of SVM algorithmHowever it may cause computational problem due to thelarge vectors and high dimensionality The idea of usingKernel function enables the calculation performed in theoriginal space instead of in the projected high dimensionedfuture space avoiding the curse of dimensionality [27 31]Given a feature mapping 120601 we define the correspondingKernel function to be

119870(119909119894 119909119895) = ⟨120601 (119909

119894) 120601 (119909

119895)⟩ (20)

Thus the linear decision hyperplane in the high featurespace is

119891 (119909) = sgn(119898

sum

119894119895=1

120572119894119910119894⟨119870 (119909

119894 119909119895)⟩ + 119887) (21)

119870(119909119894 119909119895) is inexpensive to calculate Kernel function returns

an inner production of vectors in the high feature spacewhichis evaluated in the original space not in high feature spaceThe commonly used four Kernel functions are as shown inTable 1

22 SVM Model Selection SVM algorithm is a very effectivedata classifying technique and building a model to detectfault based on SVM algorithm is not so complex Trainingdataset and testing sets are usually involved when a classi-fication is done Besides every instance in the training setcontains two parts one is target value (ie the class label)and the other is several attributes (ie observed variables)The basic process of using SVM algorithm to classify data isas follows at first build a classifier based on the training setand then use it to predict the target value of the data in testingset where only the attributes are known

To construct a classifier model and figure out faulty datathe following procedure is used

(i) Transform data collected from real process to theformat that SVM classifier can use

4 Abstract and Applied Analysis

(ii) Try a few kinds of Kernels and find out the best onethen search the optimal parameters for it This thesisuses Gaussian RBF Kernel function and the optimalparameters are obtained by using the cross-validationalgorithm

(iii) Use the optimal parameters and appropriate Kernelfunction to build a classifier

(iv) Take testing data into the constructed classifier anddo a test As a result the faulty data will be figured outand in this way the faulty station can be detected

221 RBF Kernel Which kind of Kernel functions to useis significant because it decides the feature space wheretraining data is classifiedTheoretically the Kernel function isconsidered to be better if it can provide higher upper boundon the VC dimension However it requires evaluating thehypersphere enclosing all data in the nonlinear feature spacewhich is difficult to accomplish Among the commonly usedKernel functions for the SVM algorithm the RBF Kernel ispreferable according to practical experience

Kernel function can nonlinearly project original inputdata to a higher dimensional space thus SVM equippedwith Kernel function is able to deal with the case where theinput data cannot be linearly separated Generally speakingthe RBF Kernel is the first choice for the following tworeasons Firstly linear Kernel is a special RBF Kernelmdashwhenthe parameter 119862 is adjusted to a certain value linear Kernelbehaves similarly to RBF Kernel and so it is the sigmoidKernel at some parameters Secondly the RBF Kernel bringsfewer computational costs and fewer hyperparameters [23]

222 Cross-Validation When using an RBF Kernel func-tion it needs appropriate parameters to make sure that theclassifier accurately predicts unknown data It is not knownbeforehand what the best parameter values are neverthelessthe optimal parameter searching can be accomplished usingcross-validation algorithm [23]

The cross-validation algorithm is a model validationmethod which is used to evaluate the accuracy of a predictivemodel The goal of cross-validation is to give an insight onhow the model generalizes to an independent dataset (ie anunknown dataset) by defining a dataset in the training phaseto test the model

In the process of 119896-fold cross-validation original trainingset is randomly divided into k equal size parts Sequentiallyone subset is used as the testing dataset to test the predictivemodel and the rest of 119896 minus 1 subsets are combined asthe training dataset The aforementioned validation processshould be repeated 119896 times in all with every subset performedas testing data once Using the 119896 results the predictive resultof themodel is produced and it is the overall misclassificationrate across all testing sets

This thesis uses 5-fold cross-validation algorithm to findout the overallmisclassification rate across all testing setsThecross-validation process should be performed many times topick out the parameters whichmake the overall misclassifica-tion minimise In this way the optimal parameters are foundand the classifier can obtain the best accuracy rate

23 PLS for Fault Detection ThePLS algorithm is introducedby Dayal and Macgregor [32] due to its simplicity and thelesser computational effort when dealing with the processmonitoring with large data In this technique we also denotetraining data in the form of 119898 times 119899 matrix 119883 in which 119898

represents the number of observed samples while 119899 is thequantity of the attributes and 119884 isin R119898times1 The PLS algorithmprojects thematrix119883 into a lowdimensional spacewith latentvariables and 119884 can be constructed by these latent variablesThe construct model is as follows

119883 = 119879119875119879+ 119864 =

119860

sum

119894=1

119905119894119901119879

119894+ 119864

119884 = 119879119876119879+ 119865 =

119860

sum

119894=1

119905119894119902119879

119894+ 119865

(22)

where119879 = [1199051 119905119860] is the scorematrix of119883119875 = [119901

1 119901119860]

and 119876 = [1199021 119902119860] are the loading matrix of 119883 and 119884

respectively The latent variables 119905119894isin 119879 (119894 = 1 119860) can be

directly calculated from 119883 by 119905119894= 119883119903119894 119877 = [119903

1 119903119860] and 119903

119894

are calculated by the following

1199031= 1199081

119903119894= 119908119894minus 119901119879

11199081198941199031minus 119901119879

119894minus1119908119894119903119894minus1

119894 = 2 119860

(23)

where the 119908119894(119894 = 1 119860) is the weight vector of the 119894th

deflated119883 PLS decomposes119883 into two parts

119909 = 119909 + 119909

119909 = 119875119877119879119909 isin 119878119909

119909 = (119868 minus 119875119877119879) 119909 isin 119878

119909

(24)

Usually1198792 statistic is used to detect abnormalities and thecalculation method is

1198792= 119909119879119877(

119879119879119879

119899 minus 1)

minus1

119877119879119909 (25)

With a given confidence level 120572 the threshold for 1198792 willbe calculated by the following

119869th1198792 =119860 (1198992minus 1)

119899 (119899 minus 119860)119865119897119899minus119897120572

(26)

where 119865119897119899minus119897120572

represents 119865-distribution with l and 119873 minus 1

degrees of freedom and its confidence level is 120572 If 1198792 are allless than their corresponding thresholds the process is out offault [32]

3 Simulation

31 Tennessee Eastman Process The Tennessee Eastmanprocess (TE process) is a simulation model of a practicalindustrial process The simulation model is very suitable forresearching onprocess control technology and also applicable

Abstract and Applied Analysis 5

A

D

E

C

XCXD

XEXF

FI

FI

FI

LI

LI

LISC

Ana

lyze

r

Ana

lyze

rA

naly

zer

PI

PI

PI

FI

FI

JI

TITI

TI

TI

TIFI

FI

FI

FI

FI

XAXB

XD

XE

XF

XG

XAXB

XH

XCXD

XEXFXGXH

Vapliq separator

Compressor

Purge

Product

CWS

CWS

CWR

CWR

Reactor

Stripper

Cond

Stm

Condenser

4

6

3

2

1

7

12

5

89

10

11

Figure 1 The flowsheet of TE process

for other kinds of control issues Figure 1 shows the flowsheetof TE process The TE process consists of five major partsthe reactor the product condenser a vapor-liquid separator arecycle compressor and a product stripper to help accomplishreactor separator and recycle arrangement More details canbe found in [33 34]

32 Simulated Data Intro The TE process is a plant-wideclosed-loop control structure The simulated process canproduce normal operating condition as well as 21 faulty con-ditions and generate simulated data at a sampling interval of3min For each case (no matter normal or faulty condition)two sets of data are produced training datasets and testingdatasets The training sets are used to construct statisticalpredictive model and the testing datasets are used to estimatethe accuracy of the built classifier In training sets the normaldataset contains 500 observation samples while each faultydataset contains 480 observation samples As for testing dataeach dataset (both normal and faulty conditions) consists of960 observations In the faulty condition faulty informationemerges 8h later since the TE process is turned on Thatis to say in each faulty condition the former 160 samplesare shown normally while the remaining 800 instances areshown really faulty and should be detected Every observationsample contains 52 variables which consists of 22 processmeasure variables 19 component measure variables and 12control variables All the datasets used in this paper can befound in [1]

In the following section the fault detection result of SVM-based classifier will be compared with the one based on

the PLS algorithm using the dataset generated by TE processsimulator For SVM classifier the RBF Kernel is used and therelated parameters will be set ahead according to the result ofcross-validation to make sure of the classifierrsquos high accuracyIn the comparison the performance is measured by faultdetection rates

4 Result and Discussion

TE process simulator can generate 22 types of conditionscontaining the normal condition and 21 kinds of pro-grammed faults caused by various known disturbances inthe process Once fault is added all variables will be affectedand some changes will emerge According to Chiang et al[8] and Zhang [18] detection for faults 3 9 15 and 21 isvery difficult for there are not any observable changes in themeans variance or the peak timeTherefore these four faultsalways cannot be detected by any statistics technique thusthe four faults are not analysed in this paperThe informationof all the faults is presented in Table 2

In order to profoundly test the detection effect of theclassifier based on theory of SVM the SVM algorithm andPLS algorithm are applied to the TE process respectively Asfor SVM classifier all faults data are by turns combined withthe normal condition data as the dataset for binary classifierAfter models are built by training data we use testing data toevaluate the prediction result through the following commonindices accuracy (Acc) and fault detection rate (FDR) [35]Then the detection result for each fault is shown in Table 3Table 4 represents the detection result using PLS technique

6 Abstract and Applied Analysis

Table 2 Descriptions of 21 faults in TE process

Number Fault descriptionIDV0 NormalIDV1 119860119862 feed ratio 119861 composition constantIDV2 B composition 119860119862 ratio constantIDV4 Reactor cooling water inlet temperatureIDV5 Condenser cooling water inlet temperatureIDV6 A feed loss (stream 1)IDV7 C header pressure loss-reduced availability (stream 4)IDV8 A B and C feed composition (stream 4)IDV10 C feed temperature (stream 4)IDV11 Reactor cooling water inlet temperatureIDV12 Condenser cooling water inlet temperatureIDV13 Reaction kineticsIDV14 Reactor cooling water valveIDV16 UnknownIDV17 UnknownIDV18 UnknownIDV19 UnknownIDV20 Unknown

Table 3 Results of faults detection with one against one classifier

Fault Acc () FDR ()IDV1 9844 995IDV2 9812 9812IDV4 999 9988IDV5 9198 9075IDV6 6677 6013IDV7 9958 9891IDV8 9635 96IDV10 7771 81IDV11 7302 8025IDV12 974 9775IDV13 9333 925IDV14 8906 91IDV16 8073 8938IDV17 8385 8163IDV18 901 895IDV19 7448 8588IDV20 775 805

Consider

(i) Acc =the quantity of data correctly predicated

the quantity of all data

times 100

(ii) FDR =number of faulty data correctly predicated

number of all faulty data

times 100

(27)

Table 4 Results of faults detection using PLS algorithm

Fault FDR ()1198792

1199091198792

119909

IDV1 9937 9962IDV2 9875 9812IDV4 063 9725IDV5 1665 3004IDV6 99 100IDV7 2916 100IDV8 8986 9687IDV10 1815 4355IDV11 263 6884IDV12 7685 9837IDV13 9262 9499IDV14 225 100IDV16 939 2941IDV17 488 9174IDV18 8748 8986IDV19 038 1514IDV20 184 5044

Figure 2 shows the predicted label119910 using three classifierswith the training and testing data respectively come fromnormal condition and fault 1 normal condition and fault2 and normal condition and fault 4 The first 160th testthe process shows normally and 119910 should be minus1 The faultinformation appears at the moment of 161st sample-takenand the label y should be 1 since then From Figure 2 wecan see that the SVM classifierrsquos prediction result for mostof the time is right In addition Table 3 presents detaileddetection indices of these 21 SVM classifiers It is worthmentioning that the hyperparameters of the classifiers are alloptimized beforehand Without the optimization of classifierparameter the predicted effect will not be so goodMoreoverthe index accuracy will be 1677 when using classifierrsquosdefault parameters to detect fault 1 Table 4 shows the detailedindices using PLS technique to detect faults It can be seenthat the SVM classifiers with optimal hyperparameters aremostly able to detect the faulty data and the accuracy ishigher than that given by using PLS algorithmTherefore theSVM algorithm has a better performance of fault detection

To further test the predictive ability of the SVM classifierthat is the detective ability of fault in the TE process we usenormal condition data combined with three faulty conditiondatasets (fault 1 fault 2 fault 4) as the training data toconstruct a classificationmodel and then used correspondingtest data as testing data to observe the classification result Asshown in Table 5 the detection indices are also good thoughthe computing time is a little longer with the same computefacility Thus the SVM algorithm can perform satisfactorilyon the original dataset containing 52 attributes without anytransformation In this way in real process with advancedcompute facility we can train normal data and all faults datato build a classifier Once the fault label is figured out the faultin process is detected

Abstract and Applied Analysis 7

0 200 400 600 800 1000minus2

0

2

y(0

vers

us4

)SVM(one versus one) on test data

(a)

y (0

ver

sus 2

)

0 200 400 600 800 1000minus2

0

2SVM(one versus one) on test data

(b)

0 200 400 600 800 1000minus2

0

2y

(0ve

rsus

4)

SVM(one versus one) on test data

(c)

Figure 2 Classification results for the testing dataset by the SVM-based classifier with optimal parametersThe value 119910 = minus1 represents at thesample-taken moment that the process is in normal condition while 119910 = 1 stands for faulty conditionThe top plot is the classification resultbetween normal condition and fault 1 the middle one is the result between normal condition and fault 2 and the last plot is result betweennormal condition and fault 4 In the simulation of fault condition the fault information appears from the moment of 161th sample-taken andit is pointed out by red line

Table 5 Results of faults detection with one against all classifier

Test description Test data Acc () FDR ()0 versus 1 2 IDV1 test data 9802 99620 versus 1 2 IDV2 test data 9583 9850 versus 1 2 IDV0 test data 83440 versus 1 2 4 IDV1 test data 9646 99750 versus 1 2 4 IDV2 test data 95 98880 versus 1 2 4 IDV4 test data 9698 1000 versus 1 2 4 IDV0 test data 7962

From the above-shown result of fault detection on TEprocess we can conclude that the classifier based on SVMalgorithm is of good predictive ability In addition there aretwo facts that should be mentioned First before detectingnormal condition or faulty condition we have used thetechnique of cross-validation to optimize classifierrsquos hyperpa-rametersTherefore the performance of classifier could be thebest Second the classification on TE process based on SVMalgorithm performs satisfactorily without any other processfor example foregoing data dealing process and attributesselection And this featuremakes the SVMclassifier easy to bebuilt Besides the calculation and calculate time is relativelysmall since the algorithms used are fewer

5 Conclusion

TE process a benchmark chemical engineering model isused in this paper for fault detection It can be found thatthe fault detection ability of the classifier based on SVM algo-rithm using the TE processrsquos original data is satisfactory andthis indicates the advantage of using nonlinear classification

when the number of samples or attributes is very large Bycomparing detection performancewith classifier based on thePLS algorithm the classifier based on SVM algorithm withKernel function shows superior accuracy rate In additionparameter optimization beforehand plays a great role inimproving the effectiveness of classification It also simplifiesthe problem and by using no other technique relativelydecreases the computational load to reach a satisfactoryclassification result as well

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

References

[1] S Yin Data-driven design of fault diagnosis systems [PhDdissertation] University of Duisburg-Essen 2012

[2] V N Vapnik The Nature of Statistical Learning TheorySpringer New York NY USA 1995

[3] B E Boser I M Guyon and V N Vapnik ldquoTraining algorithmfor optimal margin classifiersrdquo in Proceedings of the 5h AnnualACM Workshop on Computational Learning Theory pp 144ndash152 New York NY USA July 1992

[4] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[5] A Kulkarni V K Jayaraman and B D Kulkarni ldquoSupportvector classification with parameter tuning assisted by agent-based techniquerdquo Computers and Chemical Engineering vol 28no 3 pp 311ndash318 2004

[6] L H Chiang M E Kotanchek and A K Kordon ldquoFaultdiagnosis based on Fisher discriminant analysis and support

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 3: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

Abstract and Applied Analysis 3

where the parameter 119862 a given value is called error penaltyAs for the above-mentioned data inseparable case in order tosimplify the optimal problem define the Lagrangian to be

ℓ (119908 119887 120585 120572 120574) =1

21199082+ 119862

119898

sum

119894=1

120585119894

minus

119898

sum

119894=1

120572119894[119910119894(⟨119908 119909

119894⟩ + 119887) minus 1 + 120585

119894] minus

119898

sum

119894=1

120574119894120585119894

(10)

where 120572 120574 are the Lagrangian multipliers We consider theminimization problem as original primal problem

Consider

min119908119887120585

120579119901 (119908) = min

119908119887120585

max120572120574

ℓ (119908 119887 120585 120572 120574) (11)

When satisfying the Kuhn-Tucker condition then theprimal problem is transformed to its dual problem which is

max120572120574

120579119889(120572 120574) = max

120572120574min119908119887120585

ℓ (119908 119887 120585 120572 120574) (12)

Then the task is minimizing ℓ in (10) by adjusting thevalue of 119908 119887 120585 At the optimal point derivatives of ℓ shouldbe zero The saddle-point equation is as follows

120597ℓ

120597119908= 0 997904rArr

119898

sum

119894=1

120572119894119910119894= 0 (13)

120597ℓ

120597119887= 0 997904rArr 119908 =

119898

sum

119894=1

120572119894119910119894119909119894 (14)

120597ℓ

120597120585= 0 997904rArr 120572

119894+ 120574119894= 119862 (15)

If we take (13) (14) and (15) back into (10) we can obtainthe dual quadratic optimization problem [30 31]

max120572

119882(120572) = max120572

minus1

2

119898

sum

119894=1

119898

sum

119895=1

120572119894120572119895119910119894119910119895⟨119909119894 119909119895⟩ +

119898

sum

119896=1

120572119896

(16)

Satisfying the constrains

0 le 120572119894le 119862 119894 = 1 119898

119898

sum

119894=1

120572119894119910119894= 0

(17)

When solved the dual quadratic optimization problemshown in (16) the 120572

119894will be obtained Then take a look back

at (14) and describe the optimal 119908 using 120572119894and the form of

hyperplane can be changed to

⟨119908 119909119894⟩ + 119887 =

119898

sum

119894119895=1

120572119894119910119894⟨119909119894 119909119895⟩ + 119887 (18)

Table 1 Formulation of Kernel functions

Kernel 119870(119909119894 119909119895)

Linear 119909119879

119894sdot 119909119895

Polynomial (120574119909119879119894sdot 119909119895+ constant)119889

Gaussian RBF functions 119890minus120574||119909119894minus119909119895 ||

2

120574 constant and 119889 are kernel parameters

The classifier implementing the optimal separating hyper-plane comes out in the following form

119891 (119909) = sgn(119898

sum

119894119895=1

120572119894119910119894⟨119909119894 119909119895⟩ + 119887) (19)

However in some cases linear classifier is not suitable forexample data is overlapped or cannot be linearly separatedTherefore the input vectors should be projected into a higherdimensional feature space and there the data may be linearlyclassified more efficiently with the use of SVM algorithmHowever it may cause computational problem due to thelarge vectors and high dimensionality The idea of usingKernel function enables the calculation performed in theoriginal space instead of in the projected high dimensionedfuture space avoiding the curse of dimensionality [27 31]Given a feature mapping 120601 we define the correspondingKernel function to be

119870(119909119894 119909119895) = ⟨120601 (119909

119894) 120601 (119909

119895)⟩ (20)

Thus the linear decision hyperplane in the high featurespace is

119891 (119909) = sgn(119898

sum

119894119895=1

120572119894119910119894⟨119870 (119909

119894 119909119895)⟩ + 119887) (21)

119870(119909119894 119909119895) is inexpensive to calculate Kernel function returns

an inner production of vectors in the high feature spacewhichis evaluated in the original space not in high feature spaceThe commonly used four Kernel functions are as shown inTable 1

22 SVM Model Selection SVM algorithm is a very effectivedata classifying technique and building a model to detectfault based on SVM algorithm is not so complex Trainingdataset and testing sets are usually involved when a classi-fication is done Besides every instance in the training setcontains two parts one is target value (ie the class label)and the other is several attributes (ie observed variables)The basic process of using SVM algorithm to classify data isas follows at first build a classifier based on the training setand then use it to predict the target value of the data in testingset where only the attributes are known

To construct a classifier model and figure out faulty datathe following procedure is used

(i) Transform data collected from real process to theformat that SVM classifier can use

4 Abstract and Applied Analysis

(ii) Try a few kinds of Kernels and find out the best onethen search the optimal parameters for it This thesisuses Gaussian RBF Kernel function and the optimalparameters are obtained by using the cross-validationalgorithm

(iii) Use the optimal parameters and appropriate Kernelfunction to build a classifier

(iv) Take testing data into the constructed classifier anddo a test As a result the faulty data will be figured outand in this way the faulty station can be detected

221 RBF Kernel Which kind of Kernel functions to useis significant because it decides the feature space wheretraining data is classifiedTheoretically the Kernel function isconsidered to be better if it can provide higher upper boundon the VC dimension However it requires evaluating thehypersphere enclosing all data in the nonlinear feature spacewhich is difficult to accomplish Among the commonly usedKernel functions for the SVM algorithm the RBF Kernel ispreferable according to practical experience

Kernel function can nonlinearly project original inputdata to a higher dimensional space thus SVM equippedwith Kernel function is able to deal with the case where theinput data cannot be linearly separated Generally speakingthe RBF Kernel is the first choice for the following tworeasons Firstly linear Kernel is a special RBF Kernelmdashwhenthe parameter 119862 is adjusted to a certain value linear Kernelbehaves similarly to RBF Kernel and so it is the sigmoidKernel at some parameters Secondly the RBF Kernel bringsfewer computational costs and fewer hyperparameters [23]

222 Cross-Validation When using an RBF Kernel func-tion it needs appropriate parameters to make sure that theclassifier accurately predicts unknown data It is not knownbeforehand what the best parameter values are neverthelessthe optimal parameter searching can be accomplished usingcross-validation algorithm [23]

The cross-validation algorithm is a model validationmethod which is used to evaluate the accuracy of a predictivemodel The goal of cross-validation is to give an insight onhow the model generalizes to an independent dataset (ie anunknown dataset) by defining a dataset in the training phaseto test the model

In the process of 119896-fold cross-validation original trainingset is randomly divided into k equal size parts Sequentiallyone subset is used as the testing dataset to test the predictivemodel and the rest of 119896 minus 1 subsets are combined asthe training dataset The aforementioned validation processshould be repeated 119896 times in all with every subset performedas testing data once Using the 119896 results the predictive resultof themodel is produced and it is the overall misclassificationrate across all testing sets

This thesis uses 5-fold cross-validation algorithm to findout the overallmisclassification rate across all testing setsThecross-validation process should be performed many times topick out the parameters whichmake the overall misclassifica-tion minimise In this way the optimal parameters are foundand the classifier can obtain the best accuracy rate

23 PLS for Fault Detection ThePLS algorithm is introducedby Dayal and Macgregor [32] due to its simplicity and thelesser computational effort when dealing with the processmonitoring with large data In this technique we also denotetraining data in the form of 119898 times 119899 matrix 119883 in which 119898

represents the number of observed samples while 119899 is thequantity of the attributes and 119884 isin R119898times1 The PLS algorithmprojects thematrix119883 into a lowdimensional spacewith latentvariables and 119884 can be constructed by these latent variablesThe construct model is as follows

119883 = 119879119875119879+ 119864 =

119860

sum

119894=1

119905119894119901119879

119894+ 119864

119884 = 119879119876119879+ 119865 =

119860

sum

119894=1

119905119894119902119879

119894+ 119865

(22)

where119879 = [1199051 119905119860] is the scorematrix of119883119875 = [119901

1 119901119860]

and 119876 = [1199021 119902119860] are the loading matrix of 119883 and 119884

respectively The latent variables 119905119894isin 119879 (119894 = 1 119860) can be

directly calculated from 119883 by 119905119894= 119883119903119894 119877 = [119903

1 119903119860] and 119903

119894

are calculated by the following

1199031= 1199081

119903119894= 119908119894minus 119901119879

11199081198941199031minus 119901119879

119894minus1119908119894119903119894minus1

119894 = 2 119860

(23)

where the 119908119894(119894 = 1 119860) is the weight vector of the 119894th

deflated119883 PLS decomposes119883 into two parts

119909 = 119909 + 119909

119909 = 119875119877119879119909 isin 119878119909

119909 = (119868 minus 119875119877119879) 119909 isin 119878

119909

(24)

Usually1198792 statistic is used to detect abnormalities and thecalculation method is

1198792= 119909119879119877(

119879119879119879

119899 minus 1)

minus1

119877119879119909 (25)

With a given confidence level 120572 the threshold for 1198792 willbe calculated by the following

119869th1198792 =119860 (1198992minus 1)

119899 (119899 minus 119860)119865119897119899minus119897120572

(26)

where 119865119897119899minus119897120572

represents 119865-distribution with l and 119873 minus 1

degrees of freedom and its confidence level is 120572 If 1198792 are allless than their corresponding thresholds the process is out offault [32]

3 Simulation

31 Tennessee Eastman Process The Tennessee Eastmanprocess (TE process) is a simulation model of a practicalindustrial process The simulation model is very suitable forresearching onprocess control technology and also applicable

Abstract and Applied Analysis 5

A

D

E

C

XCXD

XEXF

FI

FI

FI

LI

LI

LISC

Ana

lyze

r

Ana

lyze

rA

naly

zer

PI

PI

PI

FI

FI

JI

TITI

TI

TI

TIFI

FI

FI

FI

FI

XAXB

XD

XE

XF

XG

XAXB

XH

XCXD

XEXFXGXH

Vapliq separator

Compressor

Purge

Product

CWS

CWS

CWR

CWR

Reactor

Stripper

Cond

Stm

Condenser

4

6

3

2

1

7

12

5

89

10

11

Figure 1 The flowsheet of TE process

for other kinds of control issues Figure 1 shows the flowsheetof TE process The TE process consists of five major partsthe reactor the product condenser a vapor-liquid separator arecycle compressor and a product stripper to help accomplishreactor separator and recycle arrangement More details canbe found in [33 34]

32 Simulated Data Intro The TE process is a plant-wideclosed-loop control structure The simulated process canproduce normal operating condition as well as 21 faulty con-ditions and generate simulated data at a sampling interval of3min For each case (no matter normal or faulty condition)two sets of data are produced training datasets and testingdatasets The training sets are used to construct statisticalpredictive model and the testing datasets are used to estimatethe accuracy of the built classifier In training sets the normaldataset contains 500 observation samples while each faultydataset contains 480 observation samples As for testing dataeach dataset (both normal and faulty conditions) consists of960 observations In the faulty condition faulty informationemerges 8h later since the TE process is turned on Thatis to say in each faulty condition the former 160 samplesare shown normally while the remaining 800 instances areshown really faulty and should be detected Every observationsample contains 52 variables which consists of 22 processmeasure variables 19 component measure variables and 12control variables All the datasets used in this paper can befound in [1]

In the following section the fault detection result of SVM-based classifier will be compared with the one based on

the PLS algorithm using the dataset generated by TE processsimulator For SVM classifier the RBF Kernel is used and therelated parameters will be set ahead according to the result ofcross-validation to make sure of the classifierrsquos high accuracyIn the comparison the performance is measured by faultdetection rates

4 Result and Discussion

TE process simulator can generate 22 types of conditionscontaining the normal condition and 21 kinds of pro-grammed faults caused by various known disturbances inthe process Once fault is added all variables will be affectedand some changes will emerge According to Chiang et al[8] and Zhang [18] detection for faults 3 9 15 and 21 isvery difficult for there are not any observable changes in themeans variance or the peak timeTherefore these four faultsalways cannot be detected by any statistics technique thusthe four faults are not analysed in this paperThe informationof all the faults is presented in Table 2

In order to profoundly test the detection effect of theclassifier based on theory of SVM the SVM algorithm andPLS algorithm are applied to the TE process respectively Asfor SVM classifier all faults data are by turns combined withthe normal condition data as the dataset for binary classifierAfter models are built by training data we use testing data toevaluate the prediction result through the following commonindices accuracy (Acc) and fault detection rate (FDR) [35]Then the detection result for each fault is shown in Table 3Table 4 represents the detection result using PLS technique

6 Abstract and Applied Analysis

Table 2 Descriptions of 21 faults in TE process

Number Fault descriptionIDV0 NormalIDV1 119860119862 feed ratio 119861 composition constantIDV2 B composition 119860119862 ratio constantIDV4 Reactor cooling water inlet temperatureIDV5 Condenser cooling water inlet temperatureIDV6 A feed loss (stream 1)IDV7 C header pressure loss-reduced availability (stream 4)IDV8 A B and C feed composition (stream 4)IDV10 C feed temperature (stream 4)IDV11 Reactor cooling water inlet temperatureIDV12 Condenser cooling water inlet temperatureIDV13 Reaction kineticsIDV14 Reactor cooling water valveIDV16 UnknownIDV17 UnknownIDV18 UnknownIDV19 UnknownIDV20 Unknown

Table 3 Results of faults detection with one against one classifier

Fault Acc () FDR ()IDV1 9844 995IDV2 9812 9812IDV4 999 9988IDV5 9198 9075IDV6 6677 6013IDV7 9958 9891IDV8 9635 96IDV10 7771 81IDV11 7302 8025IDV12 974 9775IDV13 9333 925IDV14 8906 91IDV16 8073 8938IDV17 8385 8163IDV18 901 895IDV19 7448 8588IDV20 775 805

Consider

(i) Acc =the quantity of data correctly predicated

the quantity of all data

times 100

(ii) FDR =number of faulty data correctly predicated

number of all faulty data

times 100

(27)

Table 4 Results of faults detection using PLS algorithm

Fault FDR ()1198792

1199091198792

119909

IDV1 9937 9962IDV2 9875 9812IDV4 063 9725IDV5 1665 3004IDV6 99 100IDV7 2916 100IDV8 8986 9687IDV10 1815 4355IDV11 263 6884IDV12 7685 9837IDV13 9262 9499IDV14 225 100IDV16 939 2941IDV17 488 9174IDV18 8748 8986IDV19 038 1514IDV20 184 5044

Figure 2 shows the predicted label119910 using three classifierswith the training and testing data respectively come fromnormal condition and fault 1 normal condition and fault2 and normal condition and fault 4 The first 160th testthe process shows normally and 119910 should be minus1 The faultinformation appears at the moment of 161st sample-takenand the label y should be 1 since then From Figure 2 wecan see that the SVM classifierrsquos prediction result for mostof the time is right In addition Table 3 presents detaileddetection indices of these 21 SVM classifiers It is worthmentioning that the hyperparameters of the classifiers are alloptimized beforehand Without the optimization of classifierparameter the predicted effect will not be so goodMoreoverthe index accuracy will be 1677 when using classifierrsquosdefault parameters to detect fault 1 Table 4 shows the detailedindices using PLS technique to detect faults It can be seenthat the SVM classifiers with optimal hyperparameters aremostly able to detect the faulty data and the accuracy ishigher than that given by using PLS algorithmTherefore theSVM algorithm has a better performance of fault detection

To further test the predictive ability of the SVM classifierthat is the detective ability of fault in the TE process we usenormal condition data combined with three faulty conditiondatasets (fault 1 fault 2 fault 4) as the training data toconstruct a classificationmodel and then used correspondingtest data as testing data to observe the classification result Asshown in Table 5 the detection indices are also good thoughthe computing time is a little longer with the same computefacility Thus the SVM algorithm can perform satisfactorilyon the original dataset containing 52 attributes without anytransformation In this way in real process with advancedcompute facility we can train normal data and all faults datato build a classifier Once the fault label is figured out the faultin process is detected

Abstract and Applied Analysis 7

0 200 400 600 800 1000minus2

0

2

y(0

vers

us4

)SVM(one versus one) on test data

(a)

y (0

ver

sus 2

)

0 200 400 600 800 1000minus2

0

2SVM(one versus one) on test data

(b)

0 200 400 600 800 1000minus2

0

2y

(0ve

rsus

4)

SVM(one versus one) on test data

(c)

Figure 2 Classification results for the testing dataset by the SVM-based classifier with optimal parametersThe value 119910 = minus1 represents at thesample-taken moment that the process is in normal condition while 119910 = 1 stands for faulty conditionThe top plot is the classification resultbetween normal condition and fault 1 the middle one is the result between normal condition and fault 2 and the last plot is result betweennormal condition and fault 4 In the simulation of fault condition the fault information appears from the moment of 161th sample-taken andit is pointed out by red line

Table 5 Results of faults detection with one against all classifier

Test description Test data Acc () FDR ()0 versus 1 2 IDV1 test data 9802 99620 versus 1 2 IDV2 test data 9583 9850 versus 1 2 IDV0 test data 83440 versus 1 2 4 IDV1 test data 9646 99750 versus 1 2 4 IDV2 test data 95 98880 versus 1 2 4 IDV4 test data 9698 1000 versus 1 2 4 IDV0 test data 7962

From the above-shown result of fault detection on TEprocess we can conclude that the classifier based on SVMalgorithm is of good predictive ability In addition there aretwo facts that should be mentioned First before detectingnormal condition or faulty condition we have used thetechnique of cross-validation to optimize classifierrsquos hyperpa-rametersTherefore the performance of classifier could be thebest Second the classification on TE process based on SVMalgorithm performs satisfactorily without any other processfor example foregoing data dealing process and attributesselection And this featuremakes the SVMclassifier easy to bebuilt Besides the calculation and calculate time is relativelysmall since the algorithms used are fewer

5 Conclusion

TE process a benchmark chemical engineering model isused in this paper for fault detection It can be found thatthe fault detection ability of the classifier based on SVM algo-rithm using the TE processrsquos original data is satisfactory andthis indicates the advantage of using nonlinear classification

when the number of samples or attributes is very large Bycomparing detection performancewith classifier based on thePLS algorithm the classifier based on SVM algorithm withKernel function shows superior accuracy rate In additionparameter optimization beforehand plays a great role inimproving the effectiveness of classification It also simplifiesthe problem and by using no other technique relativelydecreases the computational load to reach a satisfactoryclassification result as well

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

References

[1] S Yin Data-driven design of fault diagnosis systems [PhDdissertation] University of Duisburg-Essen 2012

[2] V N Vapnik The Nature of Statistical Learning TheorySpringer New York NY USA 1995

[3] B E Boser I M Guyon and V N Vapnik ldquoTraining algorithmfor optimal margin classifiersrdquo in Proceedings of the 5h AnnualACM Workshop on Computational Learning Theory pp 144ndash152 New York NY USA July 1992

[4] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[5] A Kulkarni V K Jayaraman and B D Kulkarni ldquoSupportvector classification with parameter tuning assisted by agent-based techniquerdquo Computers and Chemical Engineering vol 28no 3 pp 311ndash318 2004

[6] L H Chiang M E Kotanchek and A K Kordon ldquoFaultdiagnosis based on Fisher discriminant analysis and support

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 4: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

4 Abstract and Applied Analysis

(ii) Try a few kinds of Kernels and find out the best onethen search the optimal parameters for it This thesisuses Gaussian RBF Kernel function and the optimalparameters are obtained by using the cross-validationalgorithm

(iii) Use the optimal parameters and appropriate Kernelfunction to build a classifier

(iv) Take testing data into the constructed classifier anddo a test As a result the faulty data will be figured outand in this way the faulty station can be detected

221 RBF Kernel Which kind of Kernel functions to useis significant because it decides the feature space wheretraining data is classifiedTheoretically the Kernel function isconsidered to be better if it can provide higher upper boundon the VC dimension However it requires evaluating thehypersphere enclosing all data in the nonlinear feature spacewhich is difficult to accomplish Among the commonly usedKernel functions for the SVM algorithm the RBF Kernel ispreferable according to practical experience

Kernel function can nonlinearly project original inputdata to a higher dimensional space thus SVM equippedwith Kernel function is able to deal with the case where theinput data cannot be linearly separated Generally speakingthe RBF Kernel is the first choice for the following tworeasons Firstly linear Kernel is a special RBF Kernelmdashwhenthe parameter 119862 is adjusted to a certain value linear Kernelbehaves similarly to RBF Kernel and so it is the sigmoidKernel at some parameters Secondly the RBF Kernel bringsfewer computational costs and fewer hyperparameters [23]

222 Cross-Validation When using an RBF Kernel func-tion it needs appropriate parameters to make sure that theclassifier accurately predicts unknown data It is not knownbeforehand what the best parameter values are neverthelessthe optimal parameter searching can be accomplished usingcross-validation algorithm [23]

The cross-validation algorithm is a model validationmethod which is used to evaluate the accuracy of a predictivemodel The goal of cross-validation is to give an insight onhow the model generalizes to an independent dataset (ie anunknown dataset) by defining a dataset in the training phaseto test the model

In the process of 119896-fold cross-validation original trainingset is randomly divided into k equal size parts Sequentiallyone subset is used as the testing dataset to test the predictivemodel and the rest of 119896 minus 1 subsets are combined asthe training dataset The aforementioned validation processshould be repeated 119896 times in all with every subset performedas testing data once Using the 119896 results the predictive resultof themodel is produced and it is the overall misclassificationrate across all testing sets

This thesis uses 5-fold cross-validation algorithm to findout the overallmisclassification rate across all testing setsThecross-validation process should be performed many times topick out the parameters whichmake the overall misclassifica-tion minimise In this way the optimal parameters are foundand the classifier can obtain the best accuracy rate

23 PLS for Fault Detection ThePLS algorithm is introducedby Dayal and Macgregor [32] due to its simplicity and thelesser computational effort when dealing with the processmonitoring with large data In this technique we also denotetraining data in the form of 119898 times 119899 matrix 119883 in which 119898

represents the number of observed samples while 119899 is thequantity of the attributes and 119884 isin R119898times1 The PLS algorithmprojects thematrix119883 into a lowdimensional spacewith latentvariables and 119884 can be constructed by these latent variablesThe construct model is as follows

119883 = 119879119875119879+ 119864 =

119860

sum

119894=1

119905119894119901119879

119894+ 119864

119884 = 119879119876119879+ 119865 =

119860

sum

119894=1

119905119894119902119879

119894+ 119865

(22)

where119879 = [1199051 119905119860] is the scorematrix of119883119875 = [119901

1 119901119860]

and 119876 = [1199021 119902119860] are the loading matrix of 119883 and 119884

respectively The latent variables 119905119894isin 119879 (119894 = 1 119860) can be

directly calculated from 119883 by 119905119894= 119883119903119894 119877 = [119903

1 119903119860] and 119903

119894

are calculated by the following

1199031= 1199081

119903119894= 119908119894minus 119901119879

11199081198941199031minus 119901119879

119894minus1119908119894119903119894minus1

119894 = 2 119860

(23)

where the 119908119894(119894 = 1 119860) is the weight vector of the 119894th

deflated119883 PLS decomposes119883 into two parts

119909 = 119909 + 119909

119909 = 119875119877119879119909 isin 119878119909

119909 = (119868 minus 119875119877119879) 119909 isin 119878

119909

(24)

Usually1198792 statistic is used to detect abnormalities and thecalculation method is

1198792= 119909119879119877(

119879119879119879

119899 minus 1)

minus1

119877119879119909 (25)

With a given confidence level 120572 the threshold for 1198792 willbe calculated by the following

119869th1198792 =119860 (1198992minus 1)

119899 (119899 minus 119860)119865119897119899minus119897120572

(26)

where 119865119897119899minus119897120572

represents 119865-distribution with l and 119873 minus 1

degrees of freedom and its confidence level is 120572 If 1198792 are allless than their corresponding thresholds the process is out offault [32]

3 Simulation

31 Tennessee Eastman Process The Tennessee Eastmanprocess (TE process) is a simulation model of a practicalindustrial process The simulation model is very suitable forresearching onprocess control technology and also applicable

Abstract and Applied Analysis 5

A

D

E

C

XCXD

XEXF

FI

FI

FI

LI

LI

LISC

Ana

lyze

r

Ana

lyze

rA

naly

zer

PI

PI

PI

FI

FI

JI

TITI

TI

TI

TIFI

FI

FI

FI

FI

XAXB

XD

XE

XF

XG

XAXB

XH

XCXD

XEXFXGXH

Vapliq separator

Compressor

Purge

Product

CWS

CWS

CWR

CWR

Reactor

Stripper

Cond

Stm

Condenser

4

6

3

2

1

7

12

5

89

10

11

Figure 1 The flowsheet of TE process

for other kinds of control issues Figure 1 shows the flowsheetof TE process The TE process consists of five major partsthe reactor the product condenser a vapor-liquid separator arecycle compressor and a product stripper to help accomplishreactor separator and recycle arrangement More details canbe found in [33 34]

32 Simulated Data Intro The TE process is a plant-wideclosed-loop control structure The simulated process canproduce normal operating condition as well as 21 faulty con-ditions and generate simulated data at a sampling interval of3min For each case (no matter normal or faulty condition)two sets of data are produced training datasets and testingdatasets The training sets are used to construct statisticalpredictive model and the testing datasets are used to estimatethe accuracy of the built classifier In training sets the normaldataset contains 500 observation samples while each faultydataset contains 480 observation samples As for testing dataeach dataset (both normal and faulty conditions) consists of960 observations In the faulty condition faulty informationemerges 8h later since the TE process is turned on Thatis to say in each faulty condition the former 160 samplesare shown normally while the remaining 800 instances areshown really faulty and should be detected Every observationsample contains 52 variables which consists of 22 processmeasure variables 19 component measure variables and 12control variables All the datasets used in this paper can befound in [1]

In the following section the fault detection result of SVM-based classifier will be compared with the one based on

the PLS algorithm using the dataset generated by TE processsimulator For SVM classifier the RBF Kernel is used and therelated parameters will be set ahead according to the result ofcross-validation to make sure of the classifierrsquos high accuracyIn the comparison the performance is measured by faultdetection rates

4 Result and Discussion

TE process simulator can generate 22 types of conditionscontaining the normal condition and 21 kinds of pro-grammed faults caused by various known disturbances inthe process Once fault is added all variables will be affectedand some changes will emerge According to Chiang et al[8] and Zhang [18] detection for faults 3 9 15 and 21 isvery difficult for there are not any observable changes in themeans variance or the peak timeTherefore these four faultsalways cannot be detected by any statistics technique thusthe four faults are not analysed in this paperThe informationof all the faults is presented in Table 2

In order to profoundly test the detection effect of theclassifier based on theory of SVM the SVM algorithm andPLS algorithm are applied to the TE process respectively Asfor SVM classifier all faults data are by turns combined withthe normal condition data as the dataset for binary classifierAfter models are built by training data we use testing data toevaluate the prediction result through the following commonindices accuracy (Acc) and fault detection rate (FDR) [35]Then the detection result for each fault is shown in Table 3Table 4 represents the detection result using PLS technique

6 Abstract and Applied Analysis

Table 2 Descriptions of 21 faults in TE process

Number Fault descriptionIDV0 NormalIDV1 119860119862 feed ratio 119861 composition constantIDV2 B composition 119860119862 ratio constantIDV4 Reactor cooling water inlet temperatureIDV5 Condenser cooling water inlet temperatureIDV6 A feed loss (stream 1)IDV7 C header pressure loss-reduced availability (stream 4)IDV8 A B and C feed composition (stream 4)IDV10 C feed temperature (stream 4)IDV11 Reactor cooling water inlet temperatureIDV12 Condenser cooling water inlet temperatureIDV13 Reaction kineticsIDV14 Reactor cooling water valveIDV16 UnknownIDV17 UnknownIDV18 UnknownIDV19 UnknownIDV20 Unknown

Table 3 Results of faults detection with one against one classifier

Fault Acc () FDR ()IDV1 9844 995IDV2 9812 9812IDV4 999 9988IDV5 9198 9075IDV6 6677 6013IDV7 9958 9891IDV8 9635 96IDV10 7771 81IDV11 7302 8025IDV12 974 9775IDV13 9333 925IDV14 8906 91IDV16 8073 8938IDV17 8385 8163IDV18 901 895IDV19 7448 8588IDV20 775 805

Consider

(i) Acc =the quantity of data correctly predicated

the quantity of all data

times 100

(ii) FDR =number of faulty data correctly predicated

number of all faulty data

times 100

(27)

Table 4 Results of faults detection using PLS algorithm

Fault FDR ()1198792

1199091198792

119909

IDV1 9937 9962IDV2 9875 9812IDV4 063 9725IDV5 1665 3004IDV6 99 100IDV7 2916 100IDV8 8986 9687IDV10 1815 4355IDV11 263 6884IDV12 7685 9837IDV13 9262 9499IDV14 225 100IDV16 939 2941IDV17 488 9174IDV18 8748 8986IDV19 038 1514IDV20 184 5044

Figure 2 shows the predicted label119910 using three classifierswith the training and testing data respectively come fromnormal condition and fault 1 normal condition and fault2 and normal condition and fault 4 The first 160th testthe process shows normally and 119910 should be minus1 The faultinformation appears at the moment of 161st sample-takenand the label y should be 1 since then From Figure 2 wecan see that the SVM classifierrsquos prediction result for mostof the time is right In addition Table 3 presents detaileddetection indices of these 21 SVM classifiers It is worthmentioning that the hyperparameters of the classifiers are alloptimized beforehand Without the optimization of classifierparameter the predicted effect will not be so goodMoreoverthe index accuracy will be 1677 when using classifierrsquosdefault parameters to detect fault 1 Table 4 shows the detailedindices using PLS technique to detect faults It can be seenthat the SVM classifiers with optimal hyperparameters aremostly able to detect the faulty data and the accuracy ishigher than that given by using PLS algorithmTherefore theSVM algorithm has a better performance of fault detection

To further test the predictive ability of the SVM classifierthat is the detective ability of fault in the TE process we usenormal condition data combined with three faulty conditiondatasets (fault 1 fault 2 fault 4) as the training data toconstruct a classificationmodel and then used correspondingtest data as testing data to observe the classification result Asshown in Table 5 the detection indices are also good thoughthe computing time is a little longer with the same computefacility Thus the SVM algorithm can perform satisfactorilyon the original dataset containing 52 attributes without anytransformation In this way in real process with advancedcompute facility we can train normal data and all faults datato build a classifier Once the fault label is figured out the faultin process is detected

Abstract and Applied Analysis 7

0 200 400 600 800 1000minus2

0

2

y(0

vers

us4

)SVM(one versus one) on test data

(a)

y (0

ver

sus 2

)

0 200 400 600 800 1000minus2

0

2SVM(one versus one) on test data

(b)

0 200 400 600 800 1000minus2

0

2y

(0ve

rsus

4)

SVM(one versus one) on test data

(c)

Figure 2 Classification results for the testing dataset by the SVM-based classifier with optimal parametersThe value 119910 = minus1 represents at thesample-taken moment that the process is in normal condition while 119910 = 1 stands for faulty conditionThe top plot is the classification resultbetween normal condition and fault 1 the middle one is the result between normal condition and fault 2 and the last plot is result betweennormal condition and fault 4 In the simulation of fault condition the fault information appears from the moment of 161th sample-taken andit is pointed out by red line

Table 5 Results of faults detection with one against all classifier

Test description Test data Acc () FDR ()0 versus 1 2 IDV1 test data 9802 99620 versus 1 2 IDV2 test data 9583 9850 versus 1 2 IDV0 test data 83440 versus 1 2 4 IDV1 test data 9646 99750 versus 1 2 4 IDV2 test data 95 98880 versus 1 2 4 IDV4 test data 9698 1000 versus 1 2 4 IDV0 test data 7962

From the above-shown result of fault detection on TEprocess we can conclude that the classifier based on SVMalgorithm is of good predictive ability In addition there aretwo facts that should be mentioned First before detectingnormal condition or faulty condition we have used thetechnique of cross-validation to optimize classifierrsquos hyperpa-rametersTherefore the performance of classifier could be thebest Second the classification on TE process based on SVMalgorithm performs satisfactorily without any other processfor example foregoing data dealing process and attributesselection And this featuremakes the SVMclassifier easy to bebuilt Besides the calculation and calculate time is relativelysmall since the algorithms used are fewer

5 Conclusion

TE process a benchmark chemical engineering model isused in this paper for fault detection It can be found thatthe fault detection ability of the classifier based on SVM algo-rithm using the TE processrsquos original data is satisfactory andthis indicates the advantage of using nonlinear classification

when the number of samples or attributes is very large Bycomparing detection performancewith classifier based on thePLS algorithm the classifier based on SVM algorithm withKernel function shows superior accuracy rate In additionparameter optimization beforehand plays a great role inimproving the effectiveness of classification It also simplifiesthe problem and by using no other technique relativelydecreases the computational load to reach a satisfactoryclassification result as well

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

References

[1] S Yin Data-driven design of fault diagnosis systems [PhDdissertation] University of Duisburg-Essen 2012

[2] V N Vapnik The Nature of Statistical Learning TheorySpringer New York NY USA 1995

[3] B E Boser I M Guyon and V N Vapnik ldquoTraining algorithmfor optimal margin classifiersrdquo in Proceedings of the 5h AnnualACM Workshop on Computational Learning Theory pp 144ndash152 New York NY USA July 1992

[4] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[5] A Kulkarni V K Jayaraman and B D Kulkarni ldquoSupportvector classification with parameter tuning assisted by agent-based techniquerdquo Computers and Chemical Engineering vol 28no 3 pp 311ndash318 2004

[6] L H Chiang M E Kotanchek and A K Kordon ldquoFaultdiagnosis based on Fisher discriminant analysis and support

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 5: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

Abstract and Applied Analysis 5

A

D

E

C

XCXD

XEXF

FI

FI

FI

LI

LI

LISC

Ana

lyze

r

Ana

lyze

rA

naly

zer

PI

PI

PI

FI

FI

JI

TITI

TI

TI

TIFI

FI

FI

FI

FI

XAXB

XD

XE

XF

XG

XAXB

XH

XCXD

XEXFXGXH

Vapliq separator

Compressor

Purge

Product

CWS

CWS

CWR

CWR

Reactor

Stripper

Cond

Stm

Condenser

4

6

3

2

1

7

12

5

89

10

11

Figure 1 The flowsheet of TE process

for other kinds of control issues Figure 1 shows the flowsheetof TE process The TE process consists of five major partsthe reactor the product condenser a vapor-liquid separator arecycle compressor and a product stripper to help accomplishreactor separator and recycle arrangement More details canbe found in [33 34]

32 Simulated Data Intro The TE process is a plant-wideclosed-loop control structure The simulated process canproduce normal operating condition as well as 21 faulty con-ditions and generate simulated data at a sampling interval of3min For each case (no matter normal or faulty condition)two sets of data are produced training datasets and testingdatasets The training sets are used to construct statisticalpredictive model and the testing datasets are used to estimatethe accuracy of the built classifier In training sets the normaldataset contains 500 observation samples while each faultydataset contains 480 observation samples As for testing dataeach dataset (both normal and faulty conditions) consists of960 observations In the faulty condition faulty informationemerges 8h later since the TE process is turned on Thatis to say in each faulty condition the former 160 samplesare shown normally while the remaining 800 instances areshown really faulty and should be detected Every observationsample contains 52 variables which consists of 22 processmeasure variables 19 component measure variables and 12control variables All the datasets used in this paper can befound in [1]

In the following section the fault detection result of SVM-based classifier will be compared with the one based on

the PLS algorithm using the dataset generated by TE processsimulator For SVM classifier the RBF Kernel is used and therelated parameters will be set ahead according to the result ofcross-validation to make sure of the classifierrsquos high accuracyIn the comparison the performance is measured by faultdetection rates

4 Result and Discussion

TE process simulator can generate 22 types of conditionscontaining the normal condition and 21 kinds of pro-grammed faults caused by various known disturbances inthe process Once fault is added all variables will be affectedand some changes will emerge According to Chiang et al[8] and Zhang [18] detection for faults 3 9 15 and 21 isvery difficult for there are not any observable changes in themeans variance or the peak timeTherefore these four faultsalways cannot be detected by any statistics technique thusthe four faults are not analysed in this paperThe informationof all the faults is presented in Table 2

In order to profoundly test the detection effect of theclassifier based on theory of SVM the SVM algorithm andPLS algorithm are applied to the TE process respectively Asfor SVM classifier all faults data are by turns combined withthe normal condition data as the dataset for binary classifierAfter models are built by training data we use testing data toevaluate the prediction result through the following commonindices accuracy (Acc) and fault detection rate (FDR) [35]Then the detection result for each fault is shown in Table 3Table 4 represents the detection result using PLS technique

6 Abstract and Applied Analysis

Table 2 Descriptions of 21 faults in TE process

Number Fault descriptionIDV0 NormalIDV1 119860119862 feed ratio 119861 composition constantIDV2 B composition 119860119862 ratio constantIDV4 Reactor cooling water inlet temperatureIDV5 Condenser cooling water inlet temperatureIDV6 A feed loss (stream 1)IDV7 C header pressure loss-reduced availability (stream 4)IDV8 A B and C feed composition (stream 4)IDV10 C feed temperature (stream 4)IDV11 Reactor cooling water inlet temperatureIDV12 Condenser cooling water inlet temperatureIDV13 Reaction kineticsIDV14 Reactor cooling water valveIDV16 UnknownIDV17 UnknownIDV18 UnknownIDV19 UnknownIDV20 Unknown

Table 3 Results of faults detection with one against one classifier

Fault Acc () FDR ()IDV1 9844 995IDV2 9812 9812IDV4 999 9988IDV5 9198 9075IDV6 6677 6013IDV7 9958 9891IDV8 9635 96IDV10 7771 81IDV11 7302 8025IDV12 974 9775IDV13 9333 925IDV14 8906 91IDV16 8073 8938IDV17 8385 8163IDV18 901 895IDV19 7448 8588IDV20 775 805

Consider

(i) Acc =the quantity of data correctly predicated

the quantity of all data

times 100

(ii) FDR =number of faulty data correctly predicated

number of all faulty data

times 100

(27)

Table 4 Results of faults detection using PLS algorithm

Fault FDR ()1198792

1199091198792

119909

IDV1 9937 9962IDV2 9875 9812IDV4 063 9725IDV5 1665 3004IDV6 99 100IDV7 2916 100IDV8 8986 9687IDV10 1815 4355IDV11 263 6884IDV12 7685 9837IDV13 9262 9499IDV14 225 100IDV16 939 2941IDV17 488 9174IDV18 8748 8986IDV19 038 1514IDV20 184 5044

Figure 2 shows the predicted label119910 using three classifierswith the training and testing data respectively come fromnormal condition and fault 1 normal condition and fault2 and normal condition and fault 4 The first 160th testthe process shows normally and 119910 should be minus1 The faultinformation appears at the moment of 161st sample-takenand the label y should be 1 since then From Figure 2 wecan see that the SVM classifierrsquos prediction result for mostof the time is right In addition Table 3 presents detaileddetection indices of these 21 SVM classifiers It is worthmentioning that the hyperparameters of the classifiers are alloptimized beforehand Without the optimization of classifierparameter the predicted effect will not be so goodMoreoverthe index accuracy will be 1677 when using classifierrsquosdefault parameters to detect fault 1 Table 4 shows the detailedindices using PLS technique to detect faults It can be seenthat the SVM classifiers with optimal hyperparameters aremostly able to detect the faulty data and the accuracy ishigher than that given by using PLS algorithmTherefore theSVM algorithm has a better performance of fault detection

To further test the predictive ability of the SVM classifierthat is the detective ability of fault in the TE process we usenormal condition data combined with three faulty conditiondatasets (fault 1 fault 2 fault 4) as the training data toconstruct a classificationmodel and then used correspondingtest data as testing data to observe the classification result Asshown in Table 5 the detection indices are also good thoughthe computing time is a little longer with the same computefacility Thus the SVM algorithm can perform satisfactorilyon the original dataset containing 52 attributes without anytransformation In this way in real process with advancedcompute facility we can train normal data and all faults datato build a classifier Once the fault label is figured out the faultin process is detected

Abstract and Applied Analysis 7

0 200 400 600 800 1000minus2

0

2

y(0

vers

us4

)SVM(one versus one) on test data

(a)

y (0

ver

sus 2

)

0 200 400 600 800 1000minus2

0

2SVM(one versus one) on test data

(b)

0 200 400 600 800 1000minus2

0

2y

(0ve

rsus

4)

SVM(one versus one) on test data

(c)

Figure 2 Classification results for the testing dataset by the SVM-based classifier with optimal parametersThe value 119910 = minus1 represents at thesample-taken moment that the process is in normal condition while 119910 = 1 stands for faulty conditionThe top plot is the classification resultbetween normal condition and fault 1 the middle one is the result between normal condition and fault 2 and the last plot is result betweennormal condition and fault 4 In the simulation of fault condition the fault information appears from the moment of 161th sample-taken andit is pointed out by red line

Table 5 Results of faults detection with one against all classifier

Test description Test data Acc () FDR ()0 versus 1 2 IDV1 test data 9802 99620 versus 1 2 IDV2 test data 9583 9850 versus 1 2 IDV0 test data 83440 versus 1 2 4 IDV1 test data 9646 99750 versus 1 2 4 IDV2 test data 95 98880 versus 1 2 4 IDV4 test data 9698 1000 versus 1 2 4 IDV0 test data 7962

From the above-shown result of fault detection on TEprocess we can conclude that the classifier based on SVMalgorithm is of good predictive ability In addition there aretwo facts that should be mentioned First before detectingnormal condition or faulty condition we have used thetechnique of cross-validation to optimize classifierrsquos hyperpa-rametersTherefore the performance of classifier could be thebest Second the classification on TE process based on SVMalgorithm performs satisfactorily without any other processfor example foregoing data dealing process and attributesselection And this featuremakes the SVMclassifier easy to bebuilt Besides the calculation and calculate time is relativelysmall since the algorithms used are fewer

5 Conclusion

TE process a benchmark chemical engineering model isused in this paper for fault detection It can be found thatthe fault detection ability of the classifier based on SVM algo-rithm using the TE processrsquos original data is satisfactory andthis indicates the advantage of using nonlinear classification

when the number of samples or attributes is very large Bycomparing detection performancewith classifier based on thePLS algorithm the classifier based on SVM algorithm withKernel function shows superior accuracy rate In additionparameter optimization beforehand plays a great role inimproving the effectiveness of classification It also simplifiesthe problem and by using no other technique relativelydecreases the computational load to reach a satisfactoryclassification result as well

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

References

[1] S Yin Data-driven design of fault diagnosis systems [PhDdissertation] University of Duisburg-Essen 2012

[2] V N Vapnik The Nature of Statistical Learning TheorySpringer New York NY USA 1995

[3] B E Boser I M Guyon and V N Vapnik ldquoTraining algorithmfor optimal margin classifiersrdquo in Proceedings of the 5h AnnualACM Workshop on Computational Learning Theory pp 144ndash152 New York NY USA July 1992

[4] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[5] A Kulkarni V K Jayaraman and B D Kulkarni ldquoSupportvector classification with parameter tuning assisted by agent-based techniquerdquo Computers and Chemical Engineering vol 28no 3 pp 311ndash318 2004

[6] L H Chiang M E Kotanchek and A K Kordon ldquoFaultdiagnosis based on Fisher discriminant analysis and support

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 6: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

6 Abstract and Applied Analysis

Table 2 Descriptions of 21 faults in TE process

Number Fault descriptionIDV0 NormalIDV1 119860119862 feed ratio 119861 composition constantIDV2 B composition 119860119862 ratio constantIDV4 Reactor cooling water inlet temperatureIDV5 Condenser cooling water inlet temperatureIDV6 A feed loss (stream 1)IDV7 C header pressure loss-reduced availability (stream 4)IDV8 A B and C feed composition (stream 4)IDV10 C feed temperature (stream 4)IDV11 Reactor cooling water inlet temperatureIDV12 Condenser cooling water inlet temperatureIDV13 Reaction kineticsIDV14 Reactor cooling water valveIDV16 UnknownIDV17 UnknownIDV18 UnknownIDV19 UnknownIDV20 Unknown

Table 3 Results of faults detection with one against one classifier

Fault Acc () FDR ()IDV1 9844 995IDV2 9812 9812IDV4 999 9988IDV5 9198 9075IDV6 6677 6013IDV7 9958 9891IDV8 9635 96IDV10 7771 81IDV11 7302 8025IDV12 974 9775IDV13 9333 925IDV14 8906 91IDV16 8073 8938IDV17 8385 8163IDV18 901 895IDV19 7448 8588IDV20 775 805

Consider

(i) Acc =the quantity of data correctly predicated

the quantity of all data

times 100

(ii) FDR =number of faulty data correctly predicated

number of all faulty data

times 100

(27)

Table 4 Results of faults detection using PLS algorithm

Fault FDR ()1198792

1199091198792

119909

IDV1 9937 9962IDV2 9875 9812IDV4 063 9725IDV5 1665 3004IDV6 99 100IDV7 2916 100IDV8 8986 9687IDV10 1815 4355IDV11 263 6884IDV12 7685 9837IDV13 9262 9499IDV14 225 100IDV16 939 2941IDV17 488 9174IDV18 8748 8986IDV19 038 1514IDV20 184 5044

Figure 2 shows the predicted label119910 using three classifierswith the training and testing data respectively come fromnormal condition and fault 1 normal condition and fault2 and normal condition and fault 4 The first 160th testthe process shows normally and 119910 should be minus1 The faultinformation appears at the moment of 161st sample-takenand the label y should be 1 since then From Figure 2 wecan see that the SVM classifierrsquos prediction result for mostof the time is right In addition Table 3 presents detaileddetection indices of these 21 SVM classifiers It is worthmentioning that the hyperparameters of the classifiers are alloptimized beforehand Without the optimization of classifierparameter the predicted effect will not be so goodMoreoverthe index accuracy will be 1677 when using classifierrsquosdefault parameters to detect fault 1 Table 4 shows the detailedindices using PLS technique to detect faults It can be seenthat the SVM classifiers with optimal hyperparameters aremostly able to detect the faulty data and the accuracy ishigher than that given by using PLS algorithmTherefore theSVM algorithm has a better performance of fault detection

To further test the predictive ability of the SVM classifierthat is the detective ability of fault in the TE process we usenormal condition data combined with three faulty conditiondatasets (fault 1 fault 2 fault 4) as the training data toconstruct a classificationmodel and then used correspondingtest data as testing data to observe the classification result Asshown in Table 5 the detection indices are also good thoughthe computing time is a little longer with the same computefacility Thus the SVM algorithm can perform satisfactorilyon the original dataset containing 52 attributes without anytransformation In this way in real process with advancedcompute facility we can train normal data and all faults datato build a classifier Once the fault label is figured out the faultin process is detected

Abstract and Applied Analysis 7

0 200 400 600 800 1000minus2

0

2

y(0

vers

us4

)SVM(one versus one) on test data

(a)

y (0

ver

sus 2

)

0 200 400 600 800 1000minus2

0

2SVM(one versus one) on test data

(b)

0 200 400 600 800 1000minus2

0

2y

(0ve

rsus

4)

SVM(one versus one) on test data

(c)

Figure 2 Classification results for the testing dataset by the SVM-based classifier with optimal parametersThe value 119910 = minus1 represents at thesample-taken moment that the process is in normal condition while 119910 = 1 stands for faulty conditionThe top plot is the classification resultbetween normal condition and fault 1 the middle one is the result between normal condition and fault 2 and the last plot is result betweennormal condition and fault 4 In the simulation of fault condition the fault information appears from the moment of 161th sample-taken andit is pointed out by red line

Table 5 Results of faults detection with one against all classifier

Test description Test data Acc () FDR ()0 versus 1 2 IDV1 test data 9802 99620 versus 1 2 IDV2 test data 9583 9850 versus 1 2 IDV0 test data 83440 versus 1 2 4 IDV1 test data 9646 99750 versus 1 2 4 IDV2 test data 95 98880 versus 1 2 4 IDV4 test data 9698 1000 versus 1 2 4 IDV0 test data 7962

From the above-shown result of fault detection on TEprocess we can conclude that the classifier based on SVMalgorithm is of good predictive ability In addition there aretwo facts that should be mentioned First before detectingnormal condition or faulty condition we have used thetechnique of cross-validation to optimize classifierrsquos hyperpa-rametersTherefore the performance of classifier could be thebest Second the classification on TE process based on SVMalgorithm performs satisfactorily without any other processfor example foregoing data dealing process and attributesselection And this featuremakes the SVMclassifier easy to bebuilt Besides the calculation and calculate time is relativelysmall since the algorithms used are fewer

5 Conclusion

TE process a benchmark chemical engineering model isused in this paper for fault detection It can be found thatthe fault detection ability of the classifier based on SVM algo-rithm using the TE processrsquos original data is satisfactory andthis indicates the advantage of using nonlinear classification

when the number of samples or attributes is very large Bycomparing detection performancewith classifier based on thePLS algorithm the classifier based on SVM algorithm withKernel function shows superior accuracy rate In additionparameter optimization beforehand plays a great role inimproving the effectiveness of classification It also simplifiesthe problem and by using no other technique relativelydecreases the computational load to reach a satisfactoryclassification result as well

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

References

[1] S Yin Data-driven design of fault diagnosis systems [PhDdissertation] University of Duisburg-Essen 2012

[2] V N Vapnik The Nature of Statistical Learning TheorySpringer New York NY USA 1995

[3] B E Boser I M Guyon and V N Vapnik ldquoTraining algorithmfor optimal margin classifiersrdquo in Proceedings of the 5h AnnualACM Workshop on Computational Learning Theory pp 144ndash152 New York NY USA July 1992

[4] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[5] A Kulkarni V K Jayaraman and B D Kulkarni ldquoSupportvector classification with parameter tuning assisted by agent-based techniquerdquo Computers and Chemical Engineering vol 28no 3 pp 311ndash318 2004

[6] L H Chiang M E Kotanchek and A K Kordon ldquoFaultdiagnosis based on Fisher discriminant analysis and support

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 7: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

Abstract and Applied Analysis 7

0 200 400 600 800 1000minus2

0

2

y(0

vers

us4

)SVM(one versus one) on test data

(a)

y (0

ver

sus 2

)

0 200 400 600 800 1000minus2

0

2SVM(one versus one) on test data

(b)

0 200 400 600 800 1000minus2

0

2y

(0ve

rsus

4)

SVM(one versus one) on test data

(c)

Figure 2 Classification results for the testing dataset by the SVM-based classifier with optimal parametersThe value 119910 = minus1 represents at thesample-taken moment that the process is in normal condition while 119910 = 1 stands for faulty conditionThe top plot is the classification resultbetween normal condition and fault 1 the middle one is the result between normal condition and fault 2 and the last plot is result betweennormal condition and fault 4 In the simulation of fault condition the fault information appears from the moment of 161th sample-taken andit is pointed out by red line

Table 5 Results of faults detection with one against all classifier

Test description Test data Acc () FDR ()0 versus 1 2 IDV1 test data 9802 99620 versus 1 2 IDV2 test data 9583 9850 versus 1 2 IDV0 test data 83440 versus 1 2 4 IDV1 test data 9646 99750 versus 1 2 4 IDV2 test data 95 98880 versus 1 2 4 IDV4 test data 9698 1000 versus 1 2 4 IDV0 test data 7962

From the above-shown result of fault detection on TEprocess we can conclude that the classifier based on SVMalgorithm is of good predictive ability In addition there aretwo facts that should be mentioned First before detectingnormal condition or faulty condition we have used thetechnique of cross-validation to optimize classifierrsquos hyperpa-rametersTherefore the performance of classifier could be thebest Second the classification on TE process based on SVMalgorithm performs satisfactorily without any other processfor example foregoing data dealing process and attributesselection And this featuremakes the SVMclassifier easy to bebuilt Besides the calculation and calculate time is relativelysmall since the algorithms used are fewer

5 Conclusion

TE process a benchmark chemical engineering model isused in this paper for fault detection It can be found thatthe fault detection ability of the classifier based on SVM algo-rithm using the TE processrsquos original data is satisfactory andthis indicates the advantage of using nonlinear classification

when the number of samples or attributes is very large Bycomparing detection performancewith classifier based on thePLS algorithm the classifier based on SVM algorithm withKernel function shows superior accuracy rate In additionparameter optimization beforehand plays a great role inimproving the effectiveness of classification It also simplifiesthe problem and by using no other technique relativelydecreases the computational load to reach a satisfactoryclassification result as well

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

References

[1] S Yin Data-driven design of fault diagnosis systems [PhDdissertation] University of Duisburg-Essen 2012

[2] V N Vapnik The Nature of Statistical Learning TheorySpringer New York NY USA 1995

[3] B E Boser I M Guyon and V N Vapnik ldquoTraining algorithmfor optimal margin classifiersrdquo in Proceedings of the 5h AnnualACM Workshop on Computational Learning Theory pp 144ndash152 New York NY USA July 1992

[4] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[5] A Kulkarni V K Jayaraman and B D Kulkarni ldquoSupportvector classification with parameter tuning assisted by agent-based techniquerdquo Computers and Chemical Engineering vol 28no 3 pp 311ndash318 2004

[6] L H Chiang M E Kotanchek and A K Kordon ldquoFaultdiagnosis based on Fisher discriminant analysis and support

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 8: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

8 Abstract and Applied Analysis

vector machinesrdquoComputers and Chemical Engineering vol 28no 8 pp 1389ndash1401 2003

[7] K R Beebe R J Pell and M B Seasholtz Chemometrics APractical Guide Wiley New York NY USA 1998

[8] L H Chiang E L Russell and R D Braatz Fault Detection andDiagnosis in Industrial Systems Springer New York NY USA2001

[9] A Raich and A Cinar ldquoStatistical process monitoring anddisturbance diagnosis in multivariable continuous processesrdquoAIChE Journal vol 42 no 4 pp 995ndash1009 1996

[10] L H Chiang E L Russell and R D Braatz ldquoFault diagnosis inchemical processes using Fisher discriminant analysis discrim-inant partial least squares and principal component analysisrdquoChemometrics and Intelligent Laboratory Systems vol 50 no 2pp 243ndash252 2000

[11] D R Baughman Neural Networks in Bioprocessing and Chemi-cal Engineering Academic Press New York NY USA 1995

[12] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Data Mining Inference and PredictionSpringer New York NY USA 2009

[13] A S Naik S Yin S X Ding and P Zhang ldquoRecursiveidentification algorithms to design fault detection systemsrdquoJournal of Process Control vol 20 no 8 pp 957ndash965 2010

[14] S X Ding S Yin P Zhang E L Ding and A Naik ldquoAnapproach to data-driven adaptive residual generator design andimplementationrdquo in Proceedings of the 7th IFAC Symposium onFault Detection Supervision and Safety of Technical Processespp 941ndash946 Barcelona Spain July 2009

[15] S Ding P Zhang S Yin and E Ding ldquoAn integrated designframework of fault-tolerant wireless networked control systemsfor industrial automatic control applicationsrdquo IEEE Transac-tions on Industrial Informatics vol 9 no 1 pp 462ndash471 2013

[16] H Zhang Y Shi and A Mehr ldquoOn 119867infin

filtering for discrete-time takagi-sugeno fuzzy systemsrdquo IEEE Transactions on FuzzySystems vol 20 no 2 pp 396ndash401 2012

[17] H Zhang and Y Shi ldquoParameter dependent 119867infin

filteringfor linear time-varying systemsrdquo Journal of Dynamic SystemsMeasurement and Control vol 135 no 2 Article ID 0210067 7pages 2012

[18] YW Zhang ldquoEnhanced statistical analysis of nonlinear processusing KPCA KICA and SVMrdquo Chemical Engineering Sciencevol 64 pp 800ndash801 2009

[19] MMisraHH Yue S J Qin andC Ling ldquoMultivariate processmonitoring and fault diagnosis bymulti-scale PCArdquoComputersand Chemical Engineering vol 26 no 9 pp 1281ndash1293 2002

[20] S Ding S Yin K Peng H Hao and B Shen ldquoA novel schemefor key performance indicator prediction and diagnosis withapplication to an industrial hot strip millrdquo IEEE Transactionson Industrial Informatics vol 9 no 4 pp 2239ndash2247 2013

[21] B Scholkopf and A J Smola Learning with Kernels SupportVector Machines Regularization Optimization and BeyondMIT Press New York NY USA 2002

[22] N F Thornhill and A Horch ldquoAdvances and new directionsin plant-wide disturbance detection and diagnosisrdquo ControlEngineering Practice vol 15 no 10 pp 1196ndash1206 2007

[23] C W Hsu C C Chang and C Lin A Practical Guide toSupport Vector Classification Department of Computer ScienceNational Taiwan University 2010

[24] T Hastie R Tibshirani and J Friedman The Elements ofStatistical Learning Springer New York NY USA 2001

[25] J C Lagarias J A Reeds M H Wright and P E WrightldquoConvergence properties of the Nelder-Mead simplex methodin low dimensionsrdquo SIAM Journal on Optimization vol 9 no 1pp 112ndash147 1998

[26] A Kulkarni V K Jayaraman and B D Kulkarni ldquoKnowledgeincorporated support vector machines to detect faults in Ten-nessee Eastman processrdquo Computers and Chemical Engineeringvol 29 no 10 pp 2128ndash2133 2005

[27] S R Gunn Support Vector Machines for Classification andRegression Faculty of Engineering Science and MathematicsSchool of Electronics and Computer Science 1998

[28] A Widodo and B Yang ldquoSupport vector machine in machinecondition monitoring and fault diagnosisrdquo Mechanical Systemsand Signal Processing vol 21 no 6 pp 2560ndash2574 2007

[29] C J C Burges ldquoA tutorial on support vector machines forpattern recognitionrdquo Data Mining and Knowledge Discoveryvol 2 no 2 pp 121ndash167 1998

[30] V N Vapnik Statistical Learning Theory John Wiley amp SonsNew York NY USA 1998

[31] N Cristianini and J Shawe-Taylor An Introduction to SupportVector Machines Cambridge University Press 2000

[32] B S Dayal and J F Macgregor ldquoImproved PLS algorithmsrdquoJournal of Chemometrics vol 11 no 1 pp 73ndash85 1997

[33] P R Lyman and C Georgakis ldquoPlant-wide control of theTennessee Eastman problemrdquo Computers and Chemical Engi-neering vol 19 no 3 pp 321ndash331 1995

[34] J J Downs and E F Vogel ldquoA plant-wide industrial processcontrol problemrdquo Computers and Chemical Engineering vol 17no 3 pp 245ndash255 1993

[35] S Yin S X Ding A Haghani H Hao and P Zhang ldquoAcomparison study of basic data-driven fault diagnosis andprocess monitoring methods on the benchmark tennesseeeastman processrdquo Journal of Process Control vol 22 pp 1567ndash1581 2012

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of

Page 9: Research Article Study on Support Vector Machine-Based ...downloads.hindawi.com/journals/aaa/2014/836895.pdf · Research Article Study on Support Vector Machine-Based Fault Detection

Submit your manuscripts athttpwwwhindawicom

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical Problems in Engineering

Hindawi Publishing Corporationhttpwwwhindawicom

Differential EquationsInternational Journal of

Volume 2014

Applied MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Probability and StatisticsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Mathematical PhysicsAdvances in

Complex AnalysisJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

OptimizationJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

CombinatoricsHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Operations ResearchAdvances in

Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Function Spaces

Abstract and Applied AnalysisHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

International Journal of Mathematics and Mathematical Sciences

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Algebra

Discrete Dynamics in Nature and Society

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Decision SciencesAdvances in

Discrete MathematicsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014 Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014

Stochastic AnalysisInternational Journal of