Pinhole detection in steel slab images using Gabor filter and morphological features

Pinhole detection in steel slab images usingGabor filter and morphological features

Doo-chul Choi,1 Yong-ju Jeon,1 Jong Pil Yun,2 and Sang Woo Kim1,*1Division of Electrical and Computer Engineering, POSTECH, Pohang, South Korea

2Instrumentation and Control Research Group, POSCO, Pohang, South Korea

*Corresponding author: [email protected]

Received 17 February 2011; revised 24 July 2011; accepted 2 August 2011;posted 3 August 2011 (Doc. ID 142857); published 8 September 2011

Presently, product inspection for quality control is becoming an important part in the steel manufactur-ing industry. In this paper, we propose a vision-based method for detection of pinholes in the surfaceof scarfed slabs. The pinhole is a very tiny defect that is 1–5mm in diameter. Because the brightnessin the surface of a scarfed slab is not uniform and the size of a pinhole is small, it is difficult to detectpinholes. To overcome the above-mentioned difficulties, we propose a new defect detection algorithmusing a Gabor filter and morphological features. The Gabor filter was used to extract defective candi-dates. The morphological features are used to identify the pinholes among the defective candidates. Fi-nally, the experimental results show that the proposed algorithm is effective to detect pinholes in thesurface of the scarfed slab. © 2011 Optical Society of AmericaOCIS codes: 150.0150, 150.1135.

1. Introduction

Product inspection is an important step in the man-ufacture of steel to guarantee the quality of steel pro-ducts. Presently, most steps in the manufacture ofsteel are automated; however, the inspection of steelproducts is manually performed by human inspec-tors [1]. Manual inspection is time-consuming andinefficient; moreover, high detection performancecannot be achieved. Inspectors examine productsusing different criteria; therefore, manual inspectionis not sufficiently reliable or accurate. For example,manually operated quality control processes in tex-tile production have low reliability, with a detectionrate (even by highly trained inspectors) of only about70% [2]. Therefore, an automated inspection systemthat is highly reliable and accurate is required.Recently, automatic inspection systems using var-ious methods have been investigated for defect detec-tion in steel products. Defects have been detected by

eddy currents [3], ultrasonic rays, x rays [1], opticaltechniques [4], and machine vision [5–7].

In the steel manufacturing industry, variousvision-based defect inspection systems have beenintroduced. An automatic defect detection systemwith Gabor filters optimized by a univariate dynamicencoding algorithm for searches has been developedfor cracks in raw steel block [5]. A real-time vision-based defect inspection system for a coiled high-speed steel bar has been proposed [6]. Although thesesystems were developed for steel products, it wasdifficult to apply these systems directly to pinholedetection because these systems are optimized to aspecific steel type, defect type, lighting condition, andso on.

In this paper, we focus on the detection of defectsthat exist on the surface of steel slabs. Steel slabs aresemimanufactured products. The slabs tested in thispaper are used to produce steel sheets for automo-biles. Steel sheets used for automobile production re-quire high-quality specifications; hence, any defectsin the steel slabs can degrade the quality of the endproducts. Therefore, defect detection should be per-formed to enhance the quality of steel sheets and

0003-6935/11/265122-08$15.00/0© 2011 Optical Society of America

5122 APPLIED OPTICS / Vol. 50, No. 26 / 10 September 2011

to lower manufacturing costs. In particular, we ex-amine a defect called a pinhole, which is a very tinyhole on the surface of scarfed slabs. Scarfing is a pro-cess that involves burning out defective areas on thesurface of ingots or semifinished products so that theresultant product is suitable for subsequent rollingor forging. Pinholes are generated by inclusions thatare injected in the process of melting, solidification,and casting. It is difficult to detect pinholes becausethe slab surface is not uniform and the size of a pin-hole is relatively small. Both pinholes and noise com-ponents are present on a given scarfed slab surface.

To overcome the difficulties in detecting pinholes,a novel vision-based defect detection method forsteel slab surfaces is developed. In order to detectpinholes, Gabor filters were designed according totheir shape in images. However, many noise compo-nents were detected with the pinholes. To distinguisha pinhole from other noise components, masks weredesigned and morphological features were defined.Moreover, adaptive double-thresholding and size-filtering techniques were used to enhance the perfor-mance of defect detection.

The remainder of this paper is organized as fol-lows. In Section 2, we describe the configuration ofthe inspection system and analysis of the slabimages. In Section 3, we present a defect detectionalgorithm for steel slabs. In Sections 4 and 5, wereport the experimental results and conclusions,respectively.

2. Configuration of Inspection System and Analysisof Slab Images

To detect defects efficiently, high-quality images areessential. To obtain an image of the slab surface,we developed an automatic inspection system thatconsists of four components, as shown in Fig. 1. Thedefect detection module consist of a camera/lighting,frame grabber, dual-core PC, and server/monitoringsystem. The defects occur on the upper and lowersides of the slab, and the slab is wide; hence, fourcamera/lighting modules are set up for each side.Four images per side are obtained by the frame grab-ber and four dual-core PCs run the defect detectionalgorithm for two neighboring images, one image percore. The original images and defect detection resultsare sent to the server/monitoring system. The defect

detection results are displayed on the monitor in realtime, and the results are stored on the server.

Owing to the disadvantages of an area camera,such as blurring, inspection speed, and restriction ofthe inspection range [8], line scan cameras are usedin this system. The line scan cameras scan one lineeach 0:5mm vertical movement of the slab. There-fore, 20,000 lines are acquired from 10m long steelslabs. Moreover, one frame, which consists of 1000lines, are obtained from the grabber, and each lineconsists of 1024 pixels. The typical width of a slab is1:5m. The horizontal resolution of an image is about0:573mm=pixel. In general, the images of the steelslab are composed of 20 frames with a resolutionof 1024pixels × 1000pixels. One frame of a steel slabimage is shown in Fig. 2(a).

In this paper, the target to be detected is a pinhole,which can cause quality deterioration. The featuresof a defect in the image are determined by two ele-ments. These elements are the shape of the defectand the arrangement of the inspection system. Thegeometrical shape of the pinhole is shown in thecircled regions of Figs. 3(a)–3(c). The diameter of apinhole is typically 1–5mm. To detect the pinholesefficiently, light is shone at an angle of 45 ° to the sur-face of the slab. Because a hot slab radiates the redlight, we have used a horizontal array of 5W cyanLEDs as a lighting source. Light is incident on thepinhole at an inclined angle; hence, a shaded regionappears at the bottom of the pinhole. Otherwise, abright region appears at the top of the pinhole be-cause considerable light is projected and reflected.Thus, pinhole images having a vertically brightand dark intensity profile are obtained, as shownin Fig. 3(a)–3(c).

A pinhole is relatively small in size and appearssimilar to noise components. However, because ofthe arrangement of LED lighting, pinholes have anintensity that features vertical brightness and dark-ness. This is the major characteristic that distin-guishes pinholes from pseudodefects. Therefore, aGabor filter was designed that uses the intensity pro-file of pinholes. After Gabor filtering, we extractedinformation about defect candidates by binarization.To compute the intensity characteristic of pinholes,masks were designed and applied to defect candi-dates. Finally, we detected pinholes among the defectcandidates using several morphological criteria: or-ientation, relative distance, extent ratio, and arearatio.

3. Defect Detection Algorithm

A. Preprocessing

In the images from the left side and right side of aslab, there is a background that has very low graylevel values and no useful information. The horizon-tal variation of the edge line is small; hence, we useda vertical projection profile to segment a slab surfaceand background. The vertical projection profile PðxÞ[Fig. 3(b)] was obtained by taking the mean of theFig. 1. Structure of automatic inspection system.

10 September 2011 / Vol. 50, No. 26 / APPLIED OPTICS 5123

raw slab image’s pixel values along the verticaldirection:

PðxÞ ¼ 1N

XNy¼1

Iðx; yÞ; ð1Þ

where x ¼ 1; 2;…;M, y ¼ 1; 2;…;N, andM andN arewidth and the height of slab image, respectively. Thebackground, with a gray scale intensity lower than30, was removed.

After the segmentation process, the image was di-vided into subimages. To analyze the slab surface in

Fig. 2. (a) Steel slab image. (b) Vertical projection profile of (a).


detail, we partitioned the image into 10 × 8 sectionsalong the vertical and the horizontal directions, re-spectively. Subimages were obtained by choosing 2 ×2 sections such that each overlapped with neighbor-ing subimages. As a result, 63 subimages wereacquired from each segmented image, as shown inFig. 4.

B. Gabor Function

An important property of Gabor filters is that theycan extract a component corresponding to a specificscale and orientation from an image [9]. Therefore,Gabor filters have been used widely in texture ana-lysis, defect detection, and object detection [10].First, Gabor defined the one-dimensional (1D) Gaborfunction [11]. Later, Daugman extended the 1D Ga-bor function to the two-dimensional (2D) Gabor func-tion [12]. Owing to the intensity features of thepinhole image, a Gabor filter was designed to maxi-mize the magnitude of its frequency and orientationcomponents. The 2D Gabor function is a complex-valued exponential modulated by a 2D Gaussianfunction. In the spatial frequency domain, the Gaborfunction is modulated and Gaussian shifted by a

given frequency and orientation [13]. The generalform of a 2D Gabor function is given as

f ðx; yÞ ¼ 12πσxσy

exp�−

12

�x2

σ2xþ y2

σ2y

��expð2πju0xÞ: ð2Þ

The Gabor filter is split into its real and imaginaryparts. The real part of the Gabor function is suitablefor blob detection, whereas the imaginary part of theGabor function is suitable for edge detection [14]. Inthis paper, because the intensity features of pinholesare similar to edge elements, the imaginary part wasapplied to the detection of pinholes on the scarfedslab surface. The imaginary part of the 2D Gaborfunction has the general form

gðx; yÞ ¼ 12πσxσy

exp�−

12

��x0

σx

�2

þ�y0

σy

�2��

sinð2πjf x0Þ; ð3Þ

where

Fig. 3. Procedure of dividing subimages. (a) The first subimage. (b) The next subimage in the horizontal direction. (c) The next subimagein the vertical direction.

Fig. 4. Pinhole images.


x0 ¼ x cos θ þ y sin θ; y0 ¼ −x sin θ þ y cos θ:

The imaginary part of the Gabor function is definedby four parameters: f , θ, σx, and σy. The parameter fdenotes radial frequency and θ denotes orientation.σx and σy are the standard deviations of the Gaussianenvelope along the x and y axes, respectively [15]. Fora given input image Iðx; yÞ, the magnitude of theGabor-filtered output image Rðx; yÞ is obtained as

Rðx; yÞ ¼ gðx; yÞ � Iðx; yÞ

¼XM−1

m¼1

XN−1

n¼1

gðm;nÞIðx −m; y − nÞ; ð4Þ

where � denotes the convolution operation and MandN are the size of the Gabor filter. An input imageand the Gabor-filtered image are shown in Figs. 5(a)and 5(b), respectively. The magnitude response of theGabor-filtered output image is determined by fourparameters. Therefore, the parameters were chosenon the basis of the spatial frequency, orientation andsize of the pinhole. Moreover, as the size of pinholesvaries, two Gabor filters were employed to increasethe detection rate for pinholes. The parameters ofthe Gabor filters are listed in Table 1.

C. Defect Candidate Extraction

Using the Gabor-filtered image, we performed abinarization process to obtain the position and sizeinformation of the pinhole. Each slab image has

different intensity features; hence, the Gabor-filteredimage also has different intensity features. There-fore, a threshold value was determined adaptively,with a mean and variance

T ¼ meanfRðx; yÞg þ α × stdfRðx; yÞg; ð5Þ

where α is a weighting factor determined empirically.The binary image Bðx; yÞ was obtained as

If T < Rðx; yÞ then Bðx; yÞ ¼ 1Else Bðx; yÞ ¼ 0

: ð6Þ

After performing the binarization process, size filter-ing was applied to remove small components [16]. Bylabeling the binary image Bðx; yÞ, we obtained BðnÞ.To remove the incorrectly identified defect candi-dates, blobs with an area of less than four pixels wereexcluded. Based on the size and position informationof the pinhole, we identified defect candidates fromthe original image Iðx; yÞ. Each step of defect candi-date extraction is summarized in Figs. 6(a)–6(f).

D. Morphological Feature Extraction

After extracting defect candidates, morphologicalanalysis was performed to detect pinholes among thedefect candidates. Pinholes have intensity profilesthat are vertically bright and dark; hence, it was es-sential to determine the locations of the bright anddark regions for each candidate. To extract the brightand dark regions, two threshold values, Thigh andTlow, were used. They are determined by mean andstandard deviation (std) as

Thigh ¼meanfIsubðx;yÞgþ αhigh × stdfIsubðx;yÞg;Tlow ¼meanfIsubðx;yÞgþ αlow × stdfIsubðx;yÞg; ð7Þ

where αhigh and αlow are weighting factors. In thiswork, the value of the two parameters are set to3.0 and −1:5, respectively. Using the two thresholdvalues Thigh and Tlow, defect candidates can be di-vided into three parts. The bright region was definedas a binary image with pixel values greater thanThigh, and the dark region was defined as a binaryimage with pixel values less than Tlow. Bhighðx; yÞand Blowðx; yÞ were obtained as

If Thigh < Isubðx; yÞ then Bhighðx; yÞ ¼ 1 and Blowðx; yÞ ¼ 0Else Tlow > Isubðx; yÞ then Bhighðx; yÞ ¼ 0 and Blowðx; yÞ ¼ 1Else Bhighðx; yÞ ¼ 0 and Blowðx; yÞ ¼ 0

: ð8Þ

Fig. 5. Gabor filtering: (a) input image, (b) Gabor-filtered image.

Table 1. Parameters of Gabor Filters

Filter σ x σ y Frequency θ

Gabor1 1 4 1=8 π=2Gabor2 3 7 1=14 π=2


Bhighðx; yÞ is a binarized image of the bright regionand Blowðx; yÞ is a binarized image of the dark region.To evaluate the distribution of the bright and darkregions, masks were designed. The pinhole has an in-tensity profile that is sequentially bright and darkalong the vertical direction; hence, we designedmasks to maximize the score of a pinhole. The masksare shown in Figs. 7(a) and 7(b). The size of themasks was the same as that of the defect candidate.The parameters of the masks were determined sothat the maximum and minimum values, which arecalculated by multiplying the binary image andmask, would be 1 and −1, respectively. The scorewas calculated by multiplying the binarized imageand mask as follows:

If Scorehigh > 0 and Scorelow > 0then Score ¼ Scorehigh × Scorelowelse Score ¼ 0

; ð9Þ

where

Scorehigh ¼ 1MN

XMx¼1

XNy¼1

Bhighðx; yÞ ×Mhighðx; yÞ;

Scorelow ¼ 1MN

XMx¼1

XNy¼1

Blowðx; yÞ ×Mlowðx; yÞ;

where Mhigh and Mlow are masks for the bright anddark regions, respectively. To analyze morphologicalfeatures of pinholes in detail, a complete image of ablob was needed. Figure 8(b) shows a complete blobimage that contains a partial blob image [Fig. 8(a)].The white and black colors denote the bright anddark regions, respectively. From this image, we notefour morphological features: orientation, relativedistance, area ratio, and extent ratio.

1. Orientation

A pinhole has a intensity feature that is bright anddark, vertically. Therefore, the angle between thebright and dark regions can be good information toclassify the pinhole and pseudodefect. The orienta-tion is defined in Fig. 9 and is determined as

θ ¼��arctan

�yhigh − ylowxhigh − xlow

��; ð10Þ

where ðxhigh; yhighÞ and ðxlow; ylowÞ are the centroids ofthe bright and dark regions, respectively.

2. Relative Distance

As shown in Fig. 9, this feature defines how far thebright region is from the dark region. In case thebright region is far from the dark region, the defectcandidate should not be a pinhole. The relative dis-tance is calculated as

d ¼2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiðxhigh − xlowÞ2 þ ðyhigh − ylowÞ2

qffiffiffiffiffiffiffiffiffiffiffiAhigh

p þ ffiffiffiffiffiffiffiffiffiAlow

p ; ð11Þ

Fig. 6. Defect candidates extraction: (a) image after Gabor1 filtering, (b) binarized image of (a), (c) image after Gabor2 filtering,(d) binarized image of (c), (e) combined image of (b) and (c), (f) defect candidates.

Fig. 7. Proposed masks for the (a) bright and (b) dark regions.


where Ahigh and Alow are areas of the bright and darkregions in Fig. 9, respectively. Because the distancebetween the bright and dark regions increases as thesize of the pinhole becomes bigger, the value was nor-malized by the square root of the pinhole’s size.That’s why the feature was defined as relative dis-tance and not distance.

3. Area Ratio

Area ratio is defined as the ratio of area between thebright and dark regions as follows:

rarea ¼ Ahigh − Alow

Ahigh þ Alow: ð12Þ

In the case of the pinhole, the area ratio is closeto zero.

4. Extent Ratio

Extent is a scalar that specifies the ratio of pixels inthe region to pixels in the total bounding box and iscomputed as the area divided by the area of thebounding box. Extent ratio is defined as

rextent ¼12

�Ahigh

Bhighþ Alow

Blow

�; ð13Þ

where Bhigh and Blow are the area of the boundingboxes (Boxhigh and Boxlow in Fig. 9) of the brightand dark regions, respectively.

4. Experimental Results

The evaluation of the performance of the proposedalgorithm is presented in this section. We test theproposed algorithm on the slab images directly ob-tained from an product line. As shown in Fig. 2(a),

Fig. 9. Relative distance and orientation.Fig. 8. Complete blob extraction: (a) blob image of defect candi-date, (b) complete blob image that contains (a).

Fig. 10. Slab images containing pinholes: (a), (c), (e), and (g) show original images, (b), (d), (f), and (h) show images after detection.


the slab images were obtained using a line scan cam-era and had 1024 pixels × 1000pixels resolution.The diameter of a pinhole is typically 1–5mm, andthe resolutions in the horizontal and vertical axesare 0:573mm=pixel and 0:5mm=pixel, respectively.Therefore, the minimum size of a pinhole to bedetected in this experiment is 2pixels × 2pixels.The images with each processing steps are shown inFig. 6.

The proposed algorithm was tested on the 1764slab subimages, of which 116 images included pin-holes and 1648 images were defect-free. A classifica-tion algorithm such as a support vector machine(SVM) or neural network (NN) was not involved.The feature values were classified into two groups,defect and defect-free. A defect candidate in whichof all the feature values are in the defect groupwas detected as a defect. The results of detecting apinhole were shown in Fig. 10. The proposed algo-rithm has an 87.1% accuracy for the detection of pin-holes and the false positive was 3.8%. Consequently,the proposed algorithm using morphological featureswas effective at detecting pinholes in the steel slabimages.

Although the experimental results were effectiveat detecting pinholes, future work still remains to bedone. The resolution of the line scan camera shouldbe increased to obtain a more detailed image of a slabsurface. The morphological features that representthe characteristics of pinholes should be added to im-prove the performance of the proposed algorithm.Moreover, a classification algorithm such as SVMor NN should be applied to increase detectionaccuracy.

5. Conclusion

In this paper, we have proposed a new defect detec-tion algorithm. We believe pinhole detection in asteel slab surface has not been developed in previousstudies. Although there are many defect detection al-gorithms for steel products, it was hard to applythose algorithms to this application. Because the pin-hole was very small in size, it was not easy to detectthe pinhole from the slab images. To overcome thesedifficulties, we applied Gabor filters to extract defectcandidates first. Next, we defined morphological fea-tures orientation, relative distance, area ratio, andextent ratio to analyze the pinhole in detail. Imagesof steel slab surfaces were used for the performancetest of the proposed algorithm. Experimental resultsproved that the proposed defect detection algorithm

was effective for the detection of pinholes in steel slabsurfaces.

References

1. X. Li, S. K. Tso, X. P. Guan, and Q. Huang, “Improving auto-matic detection of defects in castings by applying wavelettechnique,” IEEE Trans. Ind. Electron. 53, 1927–1934 (2006).

2. C. Chan and G. K. H. Pang, “Fabric defect detection by Fourieranalysis,” IEEE Trans. Ind. Appl. 36, 1267–1276 (2000).

3. B. Helifa, A. Oulhadj, A. Benbelghit, I. K. Lefkaiser, F.Boubenider, and D. Boutassouna, “Detection and measure-ment of surface cracks in ferromagnetic materials using eddycurrent testing,” NDT&E Int. 39, 384–390 (2006).

4. L. M. Sanchez-Brea, P. Siegmann, M. A. Rebollo, and E.Bernabeu, “Optical technique for the automatic detectionand measurement of surface defects on thin metallic wires,”Appl. Opt. 39, 539–545 (2000).

5. J. P. Yun, S. H. Choi, J. W. Kim, and S. W. Kim, “Automaticdetection of cracks in raw steel block using Gabor filter opti-mized by univariate dynamic encoding algorithm for searches(uDEAS),” NDT&E Int. 42, 389–397 (2009).

6. J. P. Yun, S. H. Choi, B. Seo, and S. W. Kim, “Real-time vision-based defect inspection for high-speed steel products,” Opt.Eng. 47, 077204 (2008).

7. J. P. Yun, S. H. Choi, and S. W. Kim, “Vision-based defectdetection of scale-coverd steel billet surfaces,” Opt. Eng. 48,037205 (2009).

8. C.-S. Cho, B.-M. Chung, and M.-J. Park, “Development ofreal-time vision-based fabric inspection system,” IEEE Trans.Ind. Electron. 52, 1073–1079 (2005).

9. N. A. Kaliteevsky, V. E. Semenov, V. D. Glezer, and V. E.Gauselman, “Algorithm of invariant image description bythe use of a modified Gabor transform,” Appl. Opt. 33,5256–5261 (1994).

10. A. Kumar and G. K. H. Pang, “Defect detection in texturedmaterials using Gabor filters,” IEEE Trans. Ind. Appl. 38,425–440 (2002).

11. D. Gabor, “Theory of communication,” J. Inst. Electr. Eng.,Part 3 93, 429–441 (1946).

12. J. G. Daugman, “Uncertainty relation for resolution in space,spatial-frequency and orientation optimized by two-dimensional visual cortical filter,” J. Opt. Soc. Am. A 2,1160–1169 (1985).

13. A. Bodnarova, M. Bennamoun, and S. Latham, “OptimalGabor filters for textile flaw detection,” Pattern Recogn. 35,2973–2991 (2002).

14. D. Casasent and J. Smokelin, “Real, imaginary, and clutterGabor filter fusion for detection with reduced false alarms,”Opt. Eng. 33, 2255–2263 (1994).

15. J. Weng, J. Zhong, and C. Hu, “Phase reconstruction of digitalholography with the peak of the two-dimensional Gaborwavelet transform,” Appl. Opt. 48, 3308–3316 (2009).

16. R. C. Gonzalez and R. E. Wood, Digital Image Processing(Prentice Hall, 2002).


Documents

Pinhole detection in steel slab images using Gabor filter and morphological features