Signal classification for automaticindustrial inspection

C.B. Chittineni, M.S.E.E., Ph.D., Mem. I.E.E.E.

Indexing terms: Digital simulation, Filters, Lasers, Pattern recognition

Abstract: Details of a system designed to inspect web-type products are presented. The sensing device used isa laser scanner, and a brief description of the scanner is given. A set of bandpass filters is proposed as featureextracters. The required characteristics of the filters are determined through digital simulation, using featureselection methods. The linear classifier is designed from a set of training signals. Contextual information isused in the classification of signals into good product or into various defective categories. In addition, resultsof a study on inspection of magnetic tapes and abrasive sheets are described.

1 Introduction

Recently, automation of production line inspection tasks hasreceived increased attention, for the following reasons: (a) toreduce the manufacturing costs and the need to increase thereliability of the inspection process; (b) to satisfy the need to'make online decisions about the quality of the product,since many defects can be related to processs parameters,and manufacturing processes can be automatically controlledto improve product quality; (c) technical advances in low-costlasers [1] and LSI electronics have made inspection systemseconomically feasible for manufacturing applications.

Web-type products such as magnetic tapes, films, abrasivesetc. are manufactured at very high speeds, often at 500 feet(152 metres) or more per minute. The manufacturing of theseproducts involves a coating process on a base material andfurther operations such as curing. Many of the defects on theseproducts can be related to the parameters of the manufac-turing process. Very often the manufacturing process can beautomatically controlled to avoid making defects, if the typeof defect is identified online. In this way, the manufacturingof defective material is prevented before the product is pro-duced. Also, with online inspection, the defective material canbe marked while being manufactured. This helps in developingstrategies and systems, such as slitters, to minimise waste, toimprove quality, and to reduce cost of product manufacturing.

Pattern recognition and signal processing play a central rolein the development of systems for automatic industrial inspec-tion. This paper is concerned with the application of pattern

l_

movingweb

sensing preprocessingfeatureextraction

1 • X 2

classification

updating contextualpost-processing

i implementedI in analogue! hardware

I implementedI in software

classificationof signal

Fig. 1 Typical pattern recognition system

Paper 1580E, first received 16th December 1980 and in revised form6th July 1981The author is with Conoco Inc., PO Box 1267, Ponca City, OK 74601,USA

FEE PROC, Vol. 129, Pt. E, No. 3, MA Y 1982

recognition and signal processing [2,3] in the developmentof inspection systems. Details of a system developed for theinspection of web-type products are described; also, exper-imental results in the classification of flaws on magnetic tapesand abrasive products are presented.

2 General pattern-recognition system

The schematic of a pattern-recognition system implementedfor web inspection is shown in Fig. 1. The web is scanned byan optical scanner generating a 1-dimensional electrical signal.The signal is normalised by a preprocessing operation. Forclassification purposes, a set of features is computed by thefeature extractor. These operations are implemented in analoguehardware to meet the speed requirements of inspection. Basedon the computed features, the signal is classified as good ordefective. It is desirable to not only classify the signal but alsoto obtain its location. Down-web location is computed fromthe output of a rotopulser placed on the web. The crossweblocation is determined from the analysis of the signal.

3 Optical scanning system

The laser scanner has been found to be effective for web-typematerials. The type of laser to be used depends on the materialto be inspected. Both helium/neon and argon lasers are widelyused for inspection purposes. The schematic of a typical laserscanner is shown in Fig. 2. A brief description of the scannerfollows.

The source of light is a laser. The spot of light from thelaser is made to hit a multifaced mirror driven by a motor.

cylindrical-!lens

telescope

motor

multifaced mirror

diffused photomultiplier tubes

specular photomultipliertubes

moving web

summingamplifiers

signalprocessor

Fig. 2 Schematic of optical scanning system

0143- 7062/82/030101 + 06 $01.50/0 101

The spot reflected by the mirror hits the surface of the ma-terial to be inspected, scanning the web as the mirror isrotated. The size of the spot is controlled by a telescopemounted to the laser, and the shape of the spot is adjustedby a cylindrical lens placed in its path. The reflected lightfrom the surface of the web is either specular or diffused andis sensed by the photomultiplier tubes converting opticalenergy into electrical signals. The shape of the resulting signals,in general, contains information about the nature of thesurface to be inspected. Typical specular signals for differenttypes of defects on both magnetic tapes and abrasive sheets areshown in Fig. 3 to 5. In general, the following observationscan be made from these signals. There will be an abrupt changein the signal when the optical spot hits the edges of the pro-duct. The spacing and the number of photomultiplier tubes areadjusted in such a way that the signal, owing to the surface ofweb, is flat when there are no defects on the scan line. The

height of the signal due to the web surface when there are nodefects is called the pedestal. A defect on the scan line usuallyshows up in the shape of the signal at its correspondinglocation.

One of the important objectives of the inspection systems isto detect, not only the type of defect, but also its location onthe web. Usually, a photodiode is placed in the path of the.optical scan to generate a pulse called sync pulse for timingpurposes.

4 Preprocessing

The purpose of preprocessing is to eliminate the noise in thesignal and to normalise the signal for further processing.A photodiode placed in the path of the optical scan generatesa pulse for timing purposes. The use of preprocessing elec-tronics removes the abrupt changes in the signal caused by the

Fig. 3 Scanner signals from magnetic-tape surface

a Magnetic tape with solvent splash defectb Surface of magnetic tape with uncoated areac Magnetic tape with good surface

Fig. 4 Scanner signals from magnetic-tape surface

a Defective magnetic-tape surface with agglomerateb Defective magnetic-tape surface with agglomeratec Defective magnetic-tape surface with streaks

102 IEEPROC, Vol. 129, Pt. E, No. 3, MA Y1982

Fig. 5 Scanner signals from the surface of abrasive sheets

a Good productb Web surface with blobs defectc Web surface with wrinkles defectd Web surface with streaks defect

edges of the web and subtracts the pedestal (which is the resultof the webs surface having no defects on the scan line) fromthe signal.

5 Feature extraction

Feature extraction is one of the important parts of the system.The features to be computed, in general, are problem depend-ant (on the types of signals to be classified). Ideally, thefeatures extracted for signal classification should possess thefollowing properties:

(a) maximum discriminatory information for the classifi-cation of signals into different classes

(b) invariance to small changes in the characteristics of thesignals due to drift in the electronics, changes in the set up,optics, etc

(c) easy to implement in hardware

In practical applications, such as industrial inspection, oneof the ways of determining good features is through digitalsimulation coupled with the ingenuity and the familiarity ofthe investigator with the problem. Initially, features thatcharacterise the shape of the signals, such as peak height,width of flaw, amount of energy in the signal above a certainthreshold etc., are used with good classification results.Analogue bandpass filters are proposed as feature extractors,since they are easy to build in hardware and also meet thespeed requirements. Feature selection methods are used todetermine the characteristics of the filters through digitalsimulation. Then, active analogue bandpass filters are designedwith these characteristics and implemented in the system.

5.1 Feature selectionThis Section is concerned with the selection of m best out ofthe given n features where m <n. Let a;,-, / = 1, 2 , . . . ,M bethe pattern classes. Let X= (xx ,x2,. . . ,xn)

T be a patternvector of featuresx,-, i = 1, 2, . . . , n. Conceptually, it is de-sired to select the ra-feature combination that minimises theprobability of error as a best feature combination out of allncm feature combinations. Since the probability of error is adifficult quantity to compute numerically, divergence distancemeasure [4] is used for feature selection. The divergenceinformation /(co,-, coy) between pattern classes co,- and co,- isdefined as

*,Wy) = J [P^W-p^lWy)] ^g

(1)where p{X\tOi) are the probability density functions of thepatterns in the classes OJ,-. It is assumed that p(X\cj() areGaussian. That is,

N(Mit Z,) (2)

where Mt is the mean vector, and £,- is the convariance matrixof the patterns in the classes CJ{. Substituting eqn. 2 in eqn. 1yields

i - Mj)

(3)

For each feature combination, the means and the covarjancematrices of the training patterns in the classes are estimated.Eqn. 3 is used to compute the divergence information betweeneach pair of classes. Then either the minimum value of thedivergence information between the pairs of classes or theexpected value of the divergence information between thepairs of classes can be considered as the value of the criterionthat indicates the goodness of that feature combination. For

IEE PROC, Vol. 129, Pt. E, No. 3, MA Y1982 103

all ncm feature combinations, the value of the criterion is com-puted, and the feature combination that gives the largest valuefor the criterion is selected as the best feature subset.

Although the above procedure gives the best feature subsetwith respect to a given criterion, it is computationally expens-ive in practice, since ncm usually will be a very large number.To keep the computation practical, a suboptimal procedurecalled forward sequential feature selection is used. Initially,the features are ordered one at a time. After the single bestfeature has been selected, the remaining features are com-bined, one at a time, with the previously selected best feature,and the best two features are obtained. This procedure is thenrepeated to obtain m best features.

5.2 Filter selectionA set of analogue bandpass filters are used as feature extrac-tors for signal classification because they are easy to build inhardware and because of the inspection computational speedrequirements [5,6] . The accuracy of classification depends onthe characteristics of the filters used. Filter characteristics aredetermined through digital simulation using feature selectionmethods. Training signals from each of the defective categoriesand good products are collected. These signals are digitised andnormalised to remove the pedestal. The sampling interval isdetermined based on the highest frequency in the signals.

The approach used for filter selection is as follows. A setof central frequencies is located along the frequency axes. Thecentral frequencies chosen are shown in Fig. 6. At each fre-

Fl F2 F3 F5 F6250000 400000 550000 700000 850000 1000000

Fig. 6 Central frequencies of different filters

\

yy

80 000luO 000120 000U 0 000160 000

i uv v vFig. 7 Bandwidths of filters designed at each central frequency

quency, finite impulse response digital filters of varyingbandwidths are designed. The filter bandwidths chosen at eachcentral frequency are depicted in Fig. 7. Finite impulse re-sponse digital filters are used because they are stable and canbe made linear phase [3, 7] to prevent distortions in the filteroutputs. Training signals are passed through each filter, andfeatures are computed from the outputs of the filters. Featuresthought to have good discriminatory information for theclassification of signals are: the maximum value, the minimumvalue, and the average energy in the outputs of the filters.

Outputs of typical filters for the signals of defective mag-netic tape are illustrated in Fig. 8 to 10. From these Figures,it is noted that the signals are normalised; i.e. the pedestal isremoved from the signal. From the outputs of each filter,

Table 1: Ordered central frequencies and the selected bandwidth ateach central frequency

Order no.

123456

Central frequency

250 000850 000

1 000 000400000700000550 000

Bandwidth

160 00080 000140 00080 000160 000120 000

three features are computed: (i) the maximum value, (ii) theminimum value, and (iii) the average energy. The orderingof central frequencies and the selection of bandwidth at eachcentral frequency are done by using the feature selectionmethods discussed in the preceding Section. To meet the speedand computational requirements, analogue bandpass filters arethen designed and implemented with the computed filtercharacteristics. Table 1 shows the ordered central frequenciesand the selected bandwidth at each central frequency for theclassification of defects on abrasive sheets. The number oftraining signals used from the good, wrinkles, streaks and blobcategories are 150, 160, 170, and 150, respectively.

6 Classifier design

The type of classifier suitable for a particular applicationdepends on the distributions of patterns in the data classes.In general, points to be considered in selecting a classifier fora particular application, such as industrial inspection, are thefollowing:

(a) should be easy to implement in hardware(b) should require minimum amount of computation in

classification(c) should be easy to update classifier parameters when

more data becomes available later.

e JiW

Fig. 8 Input and outputs of typical filters for signal from surfaceof magnetic tape with streaks

Fig. 9 Input and outputs of typical filters for signal from surfaceof magnetic tape with agglomerate defect

-10

Fig. 10 Input and outputs of typical filters for signal from surfaceof magnetic tape with solvent splash defect

104 IEEPROC, Vol. 129, Pt. E, No. 3, MA Y 1982

The linear classifier is one of the simplest of all classifiers, andit is easy to implement in hardware. For these reasons, thelinear classifier is used for the classification of signals from asingle scan. For a signal, let X be the vector whose componentsare the features computed from the outputs of the filters. Foreach class /, the linear classifier computes a linear discriminantfunctiongt{X) = Wj X+coi0. A signal corresponding to apattern vector X is classified into class i if

gi(X) > gj(X) j = 1 ,2 , . . . ,M (4)

i- iwhere M is the number of classes. The parameters of thelinear discriminant functions Wt and co,0 for i = 1, 2 , . . . ,Mare computed by use of a set of training signals from eachclass [2, 8] . The first four filters of Table 1 are used as featureextractors. Three features are computed from the output ofeach of the filters. These are: maximum value, minimum value,and average energy. In practical implementation, analoguecircuits are designed to compute these features. The numberof training signals used from the good, wrinkles, streaks, andblobs categories are 75, 80, 85 and 75, respectively. Theaccuracies of classification on an independent set of testsignals are shown in Table 2. It can be seen in Table 2 that theoverall percentage of correct classification of the linear classi-fier is 75.55.

Table 2. Confusion matrix of

True Class

GoodWrinklesStreaksBlobs

classifications of set of independent testsignals

Classifier classification

Good

629

178

Wrinkles

061

96

Streaks

124

584

Blobs

161

57

6.1 Signal classification with contextual informationThe defects on the web-type materials usually run for severalscans. In general, the use of contextual information improvesthe accuracy of classification. A typical defect on a web thatextends for several scans is shown in Fig. 11.

This subsection is concerned with the use of contextualinformation [9, 10] from the past and future scans in theclassification of a signal corresponding to the current scan,(i.e., the use of contextual information from looking backand looking ahead.) Let there be M classes. Let p(u>n _ t =k\Xx, X2, . . . , Xn.x) be the a posteriori probabilities of theclasses of (n — l)th scan using all the available informationup to scan n — \. The following assumptions are made indeveloping the relationships for the use of contextualinformation.

Assumption (a)The classes of the signal of a scan, given the classes of thesignal of its previous neighboring scan and other informationfrom the previous scans, depend only on the classes of thesignal of its previous neighboring scan. i.e.

x = i,Xx,.. .,Xn.x)

P(Un = k\(X>n.y - 0

(5)

Assumption (b)

Given the true class of the signal of a scan, its feature vector

does not depend on any other information, i.e.

p(Xn\o>n = * , * , , . . . , * „ _ , ) = p(Xn\cjn = k) (6)

Assumption (c)The probability density function of the feature vector of thesignal of (n + l)th scan, given the feature vectors of the signalsof scans 1,2,. . . , n and the classes of the signal of nth scan,depends only on the classes of the signal of nth scan. i.e.

p(Xn+x\ojn = i,Xlt...,Xn) = p(Xn+l\cjn = i) (7)

It is desirable to classify the signal of the current scan, usinginformation from the previous scans. Information containedin the previous scans about the classes of the signal of thecurrent scan can be written as:

p(con = k\Xx,...,Xn.x) =

M

Y.p(un = fc, con_, = i\Xx,...,Xn.x) =M

1 = 1

)n-i) = i\Xx,...,Xn.x)P(con = fc|wn_i = /)

(8)

Using information from the feature vectors of scans 1 , 2 , . . . ,n, the a posteriori probabilities of the signal classes of thecurrent scan can be obtained as

n = k[Xlt. ..,Xn) =

M

M

, . . . , X n )

n\o3n = k)p(un = k \Xi, . . . , X W - 1 )

(9)

£ p(Xn\con = /

Information from the next scan can be used in the followingmanner in computing the a posteriori probabilities of thecurrent scan signal classes. Consider

p(Xn+1\un = k) =M

M

- j\un = k) =

p(Xn+1\u>n+i = j)P(ojn+1 = f\cjn = k) (10)

Thus,

p(con = i\Xx, . . . ,Xn,Xn + 1) -

M

y=iP(wn ~~ f>X\ , . . . , Xn+1)

p(Xn+1 \u)n = i)p(u>n = i \ X l t . . . ,Xn)M

2 p(Xn+lJ— I

,..., Xn)(11)

Eqns. 9 and 11 are applied in the following manner in theclassification of signals of L consecutive scans, using contex-tual information. The a posteriori probabilities of the first scansignal classes are updated by use of information from the first

IEE PROC, Vol. 129, Pt. E, No. 3, MA Y1982 105

Table 3: Confusion matrix of classifications of set of independent testsignals with contextual information

True class

GoodWrinklesStreaksBlobs

Classifier classification

Good

69984

Wrinkles

063

32

Streaks

64

741

Blobs

040

68

moving web

1 •

2

3-

n-1

n

f\ 1

2

3

n-1

n

H .

(uu

x , )

x2)

x3)

1 . x

x n ;

Fig. 11 Illustration of defect over several scans

and second scans. From the second through the (X-l)th scansthe updated a posteriori probabilities of the previous scansignal classes, the a posteriori probabiliites of the signal classesof the current and next scans are used in updating the aposteriori probabilities of the signal classes of the current scan.For the Lth scan, the previously updated a posteriori prob-abilities of the signal classes of (Z,-l)th scan and a posterioriprobabilities of the signal classes of Lth scan are used inupdating the a posteriori probabilities of the Z,th scan signalclasses. This procedure is repeated for every L scans. Thequantity L is referred to as the length of context.

Using the contextual information as described above, theclassification accuracies on an independent test set of signalsfrom abrasive sheets are listed in Table 3. The results ofTable 3 are obtained, choosing the value for L as four. The aposteriori probabilities of the classes are computed by normal-isation of the discriminant function values of the linear classi-fier. The following transition probabilities are used:

p(u>N = i\coc = j) =0.8 for / = /

0.2(12)

(M-l)f o r i ¥=• j

where coc is the class of the signal of the current scan, andCOJV is the class of the signal of its neighbouring scan. Table 3indicates that the overall percentage of correct classificationwith the use of contextual information is 87. ComparingTable 2 and 3, it can be seen that there is considerableimprovement in the classification accuracy with the use ofcontextual information.

7 Concluding summary

Automation of production-line inspection tasks has receivedincreased attention in recent years for the following reasons:

(a) to reduce the manufacturing costs and the need to increasethe reliability of the inspection process; (b) to satisfy theneed to make online decisions about the quality of the pro-duct, since many defects can be related to process parametersand the manufacturing processes can be controlled to improveproduct quality; (c) technical advances in low-cost lasers andLSI electronics have made inspection systems economicallyfeasible for manufacturing applications.

Details of a minicomputer-based inspection system designedto inspect web-type products are presented in this paper.A brief description of the scanner is given. The system couldbe used for inspecting web-type materials, such as magnetictapes, abrasive sheets, films, steel and printing plates etc.Products of this type are often manufactured at very highspeeds (e.g., 500 feet (152 metres) or more per minute). Thesensing device used is a laser scanner. The laser scanner gener-ates a 1-dimsensional signal. Typical scanner signals of varioustypes of defects on the surface of both magnetic tapes andabrasive sheets are presented. Signal processing and patternrecognition are used to classify signals into good and variousdefective categories. The number of scan lines per metreis 155.

A set of bandpass filters are proposed as feature extractorsto extract the features of good discriminatory information.The characteristics of these filters are determined throughdigital simulation using feature selection methods. Analoguebandpass filters having these characteristics are thenimplemented as feature extractors to meet the computationalrequirements. The linear classifier is trained to classify signalsinto different classes. The defects usually extend for severalscans. Use of contextual information has been found to behelpful in the classification of signals for visual inspection.Both 'look back' and 'look ahead' modes are implemented inusing the contextual information in' classification. Further-more, experimental results of a study for the inspection ofdefects on magnetic tapes and abrasive sheets are presented.

8 References

1 HARRY, J.E.: 'Industrial lasers and their applications' (McGraw-Hill, 1974)

2 TOU, J.T., and GONZALEZ, R.C.: 'Pattern recognition principles'(Addison-Wesley, 1974)

3 OPPENHEIM, A.V., and SCHAFER, R.W.: 'Digital signal pro-cessing' (Prentice-Hall, 1975)

4 CHITTINENI, C.B.: Efficient feature subset selection with prob-abilistic distance criteria', Inf. Sciences, 1980, 22, pp. 19-35

5 CHITTINENI, C.B.: 'Analysis of a set of band-pass filters as featureextractors for web inspection'. Technical Report, 3M Company,St. Paul, MN, Oct. 1975

6 CHITTINENI, C.B.: 'Recognition systems development programs'.Ibid., 1976

7 PARKS, T.W., and McCLELLAN, J.H.: 'A program for the designof linear phase finite impulse response filters', IEEE Trans., 1972,AU-20, pp. 195-199

8 KASHYAP, R.L.: 'Algorithms for pattern classification, inMENDEL, J.M., and FU, R.S. (Eds.) 'Adaptive, learning and patternrecognition systems' (Academic Press, New York, 1970), pp. 81 -113

9 CHITTINENI, C.B.: 'Utilization of spectral-spatial information inthe classification of imagery data'. Technical Memorandum, NASAJSC-16335, LEMSCO-14310, AgRISTARS, June 1980; Comput.Graphics & Image Process, 1981, 16, pp. 305-340

10 RAVIV, J.: 'Decision making in Markov chains applied to theproblem of pattern recognition, IEEE Trans., 1967, IT-13, pp.536-551

106 IEE PROC, Vol. 129, Pt. E, No. 3, MA Y 1982