The Development of Mechatronic Machine Vision System for Inspection Of Ceramic Plates

DESIGN AND IMPLEMENTATION OF FLEXIBLE

MANUFACTURING CELL FOR THE INSPECTION OF

CERAMIC WALL PLATESThesis submitted as partial fulfilment for MSc. degree in design and production

engineering

Ain Shams University M.U.S.T

SupervisorsProf. Farid A. Tolbah

Dr. Ahmed M. Aly

Waleed A. El-BadryT.A, Mechatronics Department,

College of Engineering, M.U.S.T

The available inspection method for quality of shapes of

drawn objects in ceramic wall plates is carried out manually.

Geometry and colours are inspected automatically in the

image as a whole irrespective of its location.

Current Inspection systems is restricted to detection of cracks

and colours only and ignores quality of shapes.

2

Manufacturing ceramic wall plates in industry is witnessing a

large growth trend, therefore a need to perform fully

automated inspection system became mandatory.

Applying an appropriate methodology for the quality

inspection of drawn shapes in ceramic plates using modified

fuzzy c-mean and fuzzy logic. Thereafter, implementing such

a methodology in a machine vision system.

Developing an automated system for inspection of ceramic

sorting system with respect to color matching and quality of

shapes

3

Eye Fatigue

accuracy of repeated measurements

lack of mass production

Since early 90s , automatic inspection played a key

rule in manufacturing . As quality may degrade with

visual inspection due to several aspect:

4

5

6

Computer Vision

ImageManipulation

ImagePresentation

ImageCompression

Storing Images

7

Source Of Illumination

Camera

Interfacing Circuitry for

Manipulators

Software Program

Mechanical Handling System

Image Processing

Machine Vision

Mechatronics

Literature Survey

Many researches in this field were not

published due to concession of production companies.

Previous research contributions were focusing on dealing with ceramic plate “as a whole” irrespective of the painted geometry of each sculpted object.

8

9

Tile Vs. Garnished Plates

Ceramic Tile

Garnished Plate

10

11

Image

Acquisition

Gamma

Correction

Image

Calibration

Soft

PartitioningColour

Grouping

Geometric

Features

Tuning Fuzzy

Logic RulesGeneral Block Diagram

12

Image Acquisition• For calibration and inspection as well.

• Acquisition speed of 60 fps with 400 MB/S (Demanded in ceramic inspection).

CCD Firewire Colour Camera

640 X 480 resolutionIEEE 1394 Frame-grabber

Image Acquisition

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping

Geometric FeaturesTuning Fuzzy Logic

Rules

13

Gamma Correctionor often simply gamma, is the name of a nonlinear operation used to code and

decode luminance in video or still images systems. This phenomena results

from the output displays. Images which are not properly corrected can look

either bleached out, or too dark

Before After

Image Acquisition

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

14

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

NEED FOR CALIBRATION ?

Camera holds a fixed number of pixels

The more pixels you use to map a feature, the better accuracy you get

Better Accuracy→ Closer Lens

15

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

DISCTORTION METRICS

16

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

Distorted Calibrated

NEED FOR CALIBRATION ?

17

Proposed Novel Algorithm For Correcting

and Calibrating Lens Distortion

Template

Acquisitio

n

Colour

Plane

Extraction

Automatic

Thresholdin

g

Particle

Groupin

g

Particle

Measuremen

ts

Displayin

g

corrected

image

Distance

Measuremen

ts

Mapping

particles to its

expected

place

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

• An Image of calibration grid is acquired.

• Red colour plane is extracted.

18



Template

Acquisitio

n

Colour

Plane

Extraction

Automatic

Thresholdin

g

Particle

Groupin

g

Particle

Measuremen

ts

Displayin

g

corrected

image

Distance

Measuremen

ts

Mapping

particles to its

expected

place

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

• Adaptive thresholding via clustering

method.

• Suitable in variable illumination over

surface.

19



Template

Acquisitio

n

Colour

Plane

Extraction

Automatic

Thresholdin

g

Particle

Groupin

g

Particle

Measuremen

ts

Displayin

g

corrected

image

Distance

Measuremen

ts

Mapping

particles to its

expected

place

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

• Tracing boundaries (checking neighbored pixels)

• Excluding open traces (incomplete circles)

20



Template

Acquisitio

n

Colour

Plane

Extraction

Automatic

Thresholdin

g

Particle

Groupin

g

Particle

Measuremen

ts

Displayin

g

corrected

image

Distance

Measuremen

ts

Mapping

particles to its

expected

place

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

21



Template

Acquisitio

n

Colour

Plane

Extraction

Automatic

Thresholdin

g

Particle

Groupin

g

Particle

Measuremen

ts

Displayin

g

corrected

image

Distance

Measuremen

ts

Mapping

particles to its

expected

place

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

• Correcting skewness (circle center, distance from

neighbors)

• Creating lookup table for correcting incoming images.

22

Distorted Image Corrected Image

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

23

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

24

Soft Partitioning and grouping

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

25

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

26

Angle measured from

reference axis

Features to be extracted from

each pattern

Features Vector for

Pattern Matching

Lines Arcs Angle

Geometry Orientation

Pixel RGB

Colour Space

ØI

wI

HIRGB

(Ri,Gi,Bi)

Geometric Feature

Extraction

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

Captured Imageafter calibration

Edge DetectionPrewitt Filter

Feature Extraction1- Corner Detection

2- Rake for Edge Measurement

3- Geometry Extraction

Geometric Feature Extraction

27

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

Classification

(Crisp)

Acceptable

Colour Spot

Colour Mismatch

28

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

Colour Quality

Deg

ree

of

mem

ber

ship

Bad Good Shiny

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

Accuracy of Geometry

Deg

ree

of

mem

ber

ship

Inaccurate Tolerated Accurate

Fuzzy

Membership

function

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

29

Fuzzy Rules


Inaccurate Tolerated Accurate

Quality

of Colour

Bad Colour Spot Colour Mismatch

Colour mismatch

Good Colour Spot Colour Spot Acceptable

Shiny Colour Spot Acceptable Acceptable

Image Acquisitio

n

Gamma Correction

Image Calibratio

n

Soft Partitionin

g

ColourGrouping


Rules

30

31

General Block Diagram

Image

AcquisitionGamma

Correction

Image

Calibration

Feature

Extraction

Evaluation of

Fuzzy RulesClassification

Colour

Matching

32

33

Mechanical DesignFor the generic FMC

34

Stations for Classification

InspectionZone

35

Flexible arms

36

Microcontroller

USB Enabled Chip

EEPROM(USB Enumeration)S

chem

atic

Dia

gra

m

37

Lay

ou

t

38

Software Architecture

MATLAB for .NET Builder

Fuzzy Logic Fuzzy C-Mean

Visual Basic.NET

Image Processing and

CalibrationUser Interface

39

40

I- Colour Spot II- Colour Mismatch III- Rotated

Fig. No.Colour Quality


Acceptable Colour Mismatch Colour Spot

I 0.61 0.68 0.29 0.0 0.5

II 0.12 0.8 0.15 0.46 0.1

III 0.92 0.85 0.59 0.0 0.05

41

Type Automatic Detection

Acceptable 95.3 %

Colour Spot 97.3%

Colour Mismatch 100 %

Total Accuracy 97.5 %

0 50 100 150 200 250 30050

60

70

80

90

Drawn Objects Orientation (in degrees)

Co

rerc

ted

Det

ecti

on

Defects Detection Rate

Effect of Object Orientation on

Defects detection

42

43

0

20

40

60

80

100

120

Colour spots Colour Mismatch Geometry

1

2

1Ovidiu Ghita, Tim Carew and Paul Whelan, A vision-based system for

inspecting painted slates, Journal of Sensor Review, Vol. 26, No.2, 2006

2Proposed Algorithm

44

Fuzzy C-Means

Correct Match(94%)

Incorrect Match(6%)

Nearest Neighbour

Correct Match(67%)

Incorrect Match(33%)

45

Criteria Time (ms)

Image acquisition 16.67

Gamma correction 23

Image calibration 86

Prewitt filter 15

Geometric feature extraction 120

Fuzzy clustering 15 Sec

Color Matching 80

Evaluation of fuzzy rules 69

Criteria Specification

USB actual transfer speed 1MB / S

Inspection Area 20 X 10 cm2

46

47

A new visualization approach for colour grouping was also proposed using fuzzy c-mean clustering technique.

A proposed calibration algorithm was to correct the lens distortion by means of software.

A fuzzy inference engine was built to classify the garnished ceramic plates into three common categories (acceptable, ceramic plate having colour spots, and ceramic plate having colour mismatched drawn objects)

48

The system shows promising results in terms of accuracy in correct classification and withstanding against variability in illumination distribution.

A test rig was developed to emulate the production environment.

The system is considered novel compared to other published work since it is the only work which considered geometric features of drawn objects up to the time of submitting this thesis.

49

[1] A. M. Aly and W. A. El-Badry, "Design and Implementation of Flexible Manufacturing Cell for Quality Inspection of Garnished Ceramic Wall Plates" in 19th

Conference of French Congress of Mechanics Marseille, France, 2009.

[2] F. A. Tolbah, A. M. Aly, and W. A. El-Badry, "Automated grading system for garnished wall plates: A mechatronic approach" presented at the 8th International Conference on Production Engineering and Design for Development, Cairo, Egypt, 2010.

[3] F. A. Tolba, A. M. Aly, and W. A. El-Badry, "An Enhanced Vision System for Sorting Ceramic Plates Based on Hybrid Algorithm and USB Interfacing Circuitry" presented at the 27th National Radio Science Conference, Menoufia, Egypt, 2010.

50

Supervisors

FAMILY

COLLEAGUES

MY BEST FRIEND

REFEREES

51

Education

The Development of Mechatronic Machine Vision System for Inspection Of Ceramic Plates