IntroductionColour SpaceClassificationFuzzy Contrast FusionRule ExtractionConclusionsLearningLearning D.Playne, V.Mehta, N.Reyes, A.Barczak Computer Science,

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearning

LearningLearning

D.Playne, V.Mehta, N.Reyes, A.BarczakD.Playne, V.Mehta, N.Reyes, A.BarczakComputer Science, Massey University, Auckland, New ZealandComputer Science, Massey University, Auckland, New Zealand

anandd

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningIntroductionIntroduction

• Recognize robots via their Recognize robots via their colour patchescolour patches and track them in and track them in real-timereal-time

• Perform colour corrections for object recognition adaptively using Perform colour corrections for object recognition adaptively using Fuzzy LogicFuzzy Logic

• LearnLearn colour descriptors using successive frames colour descriptors using successive frames

• Extract the Extract the best fuzzy rulesbest fuzzy rules for any given colour automatically for any given colour automatically


Source of Pictures: http://luminous-landscape.com/tutorials/color_and_vision.shtml




Spectral sensitivities of the RED, GREEN and BLUE CONES



The colour sensed is the result of a combination of a The colour sensed is the result of a combination of a multitude of factors.multitude of factors.

Eye

Light SourceObject

Fuzzy Colour Contrast FusionFuzzy Colour Contrast Fusion


UNDER BLUISH ILLUMINANT UNDER YELLOWISH ILLUMINANT

fundamental characteristic of the visual system which compensates for changes in illuminant color in order to keep object colours stable


a prioria priori knowledge of colour locus shifting in different illuminations

Adaptive contrast Adaptive contrast operationsoperations

+

Fuzzy Inference SystemFuzzy Inference System

Humans tend to remember colours of familiar objects

Different cones have different levels of sensitivity to Red,

Green & Blue


Colour objects

Fluorescent lampsOverhead Camera

Exploratory environment is indoor – room totally obstructed from sunlight

Multiple monochromatic light sources – fluorescent / fluoride lamps

Colour Object Recognition – speed is < 33ms, Camera height is approx. 2m.

www.Fira.net


We need to automatically compensate for theWe need to automatically compensate for theeffects of varying illumination intensities in effects of varying illumination intensities in the scene of traversalthe scene of traversal

**

Dark

Bright

Dim

Lens focus

Object rotation

Quantum electrical effects

Shadows

Presence of similar colours

Other Factors:


Color is not captured by the camera as we

humans see it.

Yellow object turns pale under strong white illumination

A Green object tends to appear more as a whitish yellow object under bright

white illumination.

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningColour SpaceColour Space

Origin moved to the location of white.

O r

gPie slice color

decision region

Pie-slice decision region was originally devised by Thomas et al.; however, they utilized the UV space. In this research, we transformed the rg-space to utilize the same pie-slice decision region.

RadiusRadius: rg-Saturation

AngleAngle: rg-Hue

New Color attributes:

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningColour SpaceColour Space

The new color descriptors, namely rg-Saturation and rg-Hue are derived as follows:

1. Compute rg-chromaticities from the camera RGB tristimulus:

2. Assign white as the new reference point or origin (Onew) of the new color space Onew(0.333, 0.333), then compute for rg-Saturation as the radius extending from the origin to any given pair of rg-chromaticities.

3. Next, compute for the rg-Hue descriptor as the angle relative to the Xnew-axis, with origin at Onew:

BGR

Rr

22 )333.0()333.0( grSaturationrg

))333.0/()333.0((tan 1 rgHuerg

BGR

Gg

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningClassificationClassification

IfIf (rg-Saturation >= rmin) and (rg-Saturation <= rmax)

(rg-Hue >= θ1 ) and (rg-Hue <= θ2) ThenThen Color = Pink.

Chrominance Point

Color decision region for Color 1

rg-Saturation

Xnew

1

2

Gray zone

rgray zone

Ynew

θ (Rg-Hue)

r

g Color decision region

Onew

Pie-slice decision region was originally devised by Thomas et al.; however, they utilized the UV space. In this research, we transformed the rg-space to utilize the same pie-slice decision region.

e.g. PINK

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningClassificationClassification

Would the pie-slice decision region suffice for accurate color object identification in the newly transformed rg-chromaticity space?

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningFuzzy Contrast FusionFuzzy Contrast Fusion

Light Blue Patches

Scene confounded with spatially

varying illumination and highlights

Presence of similarly coloured objects (i.e.

Light Blue, Blue & Violet)

Let’s zero-in on some of the colour identification targets…


Object 1 Object 2 Object 3 Object 4

Object 5 Object 6 Object 7 Objects 1-to-7

r

gr

g

r

gr

g

r

gr

g

r

g

1 2 3 4

5 6 7 All

r

g


There is a need to adaptively apply There is a need to adaptively apply contrast operations on each of the contrast operations on each of the colors comprising an object to colors comprising an object to localize localize the color locus formationthe color locus formation at some fixed at some fixed region.region.

Pie-Slice Decision RegionPie-Slice Decision Region


CONTRAST ENHANCEMENT

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1.000

0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000

RAW SIGNAL

OU

TP

UT

SIG

NA

L SIGMOID 1X

SIGMOID 2X

SIGMOID 3X

SIGMOID 4X

SIGMOID 5X

SIGMOID 6X

Excel File

Source: T. Ross, Fuzzy Logic with Engineering Applications. Singapore: McGraw-Hill, Inc., (1997).


This operation acts in a combination of contraction and This operation acts in a combination of contraction and dilation. dilation.

)(2 2 y

2)](1[21 y for 0 <= μα(y) <= 0.5

for 0.5 <= μα(y) <= 1

Source: T. Ross, Fuzzy Logic with Engineering Applications. Singapore: McGraw-Hill, Inc., (1997).

Pal, S., and D. Majumder. (1986). In Fuzzy mathematical approach to pattern recognition, John Wiley & Sons, New York.

Contrast Enhance: Logistic FunctionContrast Enhance: Logistic Function


1871120158158

2374107166166

3156112166153

2651115117122

3851569971

1871120158158

2374107166166

3156112166153

2651115117122

3851569971

1871120158158

2374107166166

3156112166153

2651115117122

3851569971

Red ComponentsGreen Components

Blue Components

Process each colour Process each colour channel in isolationchannel in isolation

Image


77 89 77 64 77 71 99 56 51 38

77 122 125 125 125 122 117 115 51 26

97 115 140 135 133 153 166 112 56 31

82 112 145 130 150 166 166 107 74 23

84 107 140 138 135 158 158 120 71 18

77 110 143 148 133 145 148 122 77 13

79 102 99 102 97 94 92 115 77 18

71 77 77 77 71 64 77 89 51 20

64 64 48 51 51 38 51 31 26 18

51 38 26 26 31 13 26 26 26 13

Value of BlueBlue component

ExampleExample:: Enhancing the Blue component of a pixelEnhancing the Blue component of a pixel


0.3 0.3 0.3 0.3 0.3 0.3 0.4 0.2 0.2 0.1

0.3 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.2 0.1

0.4 0.5 0.5 0.5 0.5 0.6 0.7 0.4 0.2 0.1

0.3 0.4 0.6 0.5 0.6 0.7 0.7 0.4 0.3 0.1

0.3 0.4 0.5 0.5 0.5 0.6 0.6 0.5 0.3 0.1

0.3 0.4 0.6 0.6 0.5 0.6 0.6 0.5 0.3 0.1

0.3 0.4 0.4 0.4 0.4 0.4 0.4 0.5 0.3 0.1

0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.2 0.1

0.3 0.3 0.2 0.2 0.2 0.1 0.2 0.1 0.1 0.1

0.2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1

0.45098

Scaled Component =

115/255



0.2 0.2 0.2 0.1 0.2 0.2 0.3 0.1 0.1 0

0.2 0.5 0.5 0.5 0.5 0.5 0.4 0.4 0.1 0

0.3 0.4 0.6 0.6 0.5 0.7 0.8 0.4 0.1 0

0.2 0.4 0.6 0.5 0.7 0.8 0.8 0.4 0.2 0

0.2 0.4 0.6 0.6 0.6 0.7 0.7 0.4 0.2 0

0.2 0.4 0.6 0.6 0.5 0.6 0.6 0.5 0.2 0

0.2 0.3 0.3 0.3 0.3 0.3 0.3 0.41 0.2 0

0.2 0.2 0.2 0.2 0.2 0.1 0.2 0.2 0.1 0

0.1 0.1 0.1 0.1 0.1 0 0.1 0 0 0

0.1 0 0 0 0 0 0 0 0 0

Enhanced Component

(Normalized)0.4067666


NORMALIZED COLOR COMPONENT MATRIX AFTER APPLYING THE ENHANCEMENT ALGORITHM


Enhanced component (ready for display)

47 62 47 32 47 40 77 25 20 11

47 117 123 123 123 117 107 104 20 5.3

74 104 151 142 138 173 193 98 25 7.5

53 98 160 132 169 193 193 90 43 4.1

55 90 151 148 142 181 181 113 40 2.5

47 95 157 165 138 160 165 117 47 1.3

49 82 77 82 74 69 66 104 47 2.5

40 47 47 47 40 32 47 62 20 3.1

32 32 18 20 20 11 20 7.5 5.3 2.5

20 11 5.3 5.3 7.5 1.3 5.3 5.3 5.3 1.3



Raw Image Enhanced Image


CONTRAST DEGRADATION

0.000

0.100

0.200

0.300

0.400

0.500

0.600

0.700

0.800

0.900

1.000

0.0000 0.1000 0.2000 0.3000 0.4000 0.5000 0.6000 0.7000 0.8000 0.9000 1.0000

RAW SIGNAL

OU

TP

UT

SIG

NA

L LOGIT 1X

LOGIT 2X

LOGIT 3X

LOGIT 4X

LOGIT 5X

LOGIT 6X

Excel File


2]5.0)([25.0 y

Contrast Degrade: Logit FunctionContrast Degrade: Logit Function

This operation pulls input signals towards 0.5This operation pulls input signals towards 0.5

for 0 <= μα(y) <= 0.5

for 0.5 <= μα(y) <= 15.0)]]]5.0)([1[2(1[ 2 y


• Performed Performed separatelyseparately on eachon each of the RGB color values of the RGB color values

• Acts on either one of the following operations: Enhance, Degrade or Retain original Acts on either one of the following operations: Enhance, Degrade or Retain original valuevalue

• Simultaneously accounts for all confounding factors found in natural scenes.Simultaneously accounts for all confounding factors found in natural scenes.

Degrade

Enhance Retain

Colour Channelenhanceenhancedegradedegraderetainretain


rg-chrom. space

Rg-Hue,

Rg-Sat

rg-chrom. space

Rg-Hue, Rg-Sat

RNEW, GNEW, BNEW

Perceived Color

(Pie-Slice Decision Region)

Adaptive Color Contrast Operations

Color Contrast Fusion

Cam

era RG

B

Target color

Contrast Constraints

This algorithm appears on: N. H. Reyes, E. P. Dadios,. "Dynamic Color Object Recognition", Int’l. Journal Of Advanced Computational Intelligence, Feb. 2004, Japan


Pie-slice decisionLight Blue

Blue

Violet

…etc.

R,

G,

B

Rules for Red

Red

Rules for Green

Green

Rules for Blue

Blue

New Red

New Green

New Blue

Degrade

Enhance

Retain

Degrade

Enhance

Retain

Degrade

Enhance

Retain

Rg

-Hu

e, R

g-S

atu

rati

on

INP

UT

SIN

PU

TS

(fr

om

Cam

era)

OU

TP

UT

SO

UT

PU

TS

(C

orr

ec

ted

Co

lou

r V

alu

es)

Colour Contrast Constraints

Target Colour


If (rg-Hue depicts Light Blue) (rg-Hue depicts Light Blue)

Then

Apply Apply HighHigh Contrast Contrast DegradeDegrade on on Red channel and and

Apply Apply LowLow Contrast Contrast EnhanceEnhance on on Green channel and and

Apply Apply MediumMedium Contrast Contrast DegradeDegrade on on Blue Channel..

Empirically derived!

ExampleExample:: Colour Contrast Rule for Light BlueColour Contrast Rule for Light Blue


Pie-slice decision region + ContrastPie-slice decision region

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningLearningLearning

Initial Parameters

Calibrated Parameters

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearningLearningLearning


The centre and radius of the circle has now been found so the next part of the algorithm can run. The learning algorithm works on a moving average system combined with a decaying learning rate algorithm. The algorithm will run for a set number of iterations and keep moving average of the maximum and minimum rg-Hue and rg-Saturation:

LearningLearning


The idea of the algorithm is to move a robot with a colour patch or roll a ballaround the board to calibrate the colour. Because the object will move through all of the different illumination conditions, the algorithm will calibrate the colour classifier to work for the entire board, accounting for all possible illumination conditions.

LearningLearning

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearning Rule ExtractionRule Extraction



Colour Name R G B Score Hits Misses comments

1 Yellow 3 1 -2 0.64853 2104 68 all objects identified

2 Green 0 -1 -3 0.552422 3313 383 all objects identified

3 Pink 1 -1 0 0.586446 1714 99 all objects identified

4 Purple 0 1 -3 0.572888 2777 314 all objects identified

5 Violet 1 1 2 0.526654 2535 497 all objects identified

6 Light Blue 0 3 1 0.668808 2758 68 all objects identified

Colour Name Rank R G B Score Hits Misses

1 Yellow 0th 0 2 -2 0.480738 2410 458

2 Green 8th -1 2 -2 0.454434 3252 608

3 Pink 4th 1 -1 0 0.586446 1714 99

4 Purple 3rd 1 1 0 0.544609 2629 320

5 Violet 0th 0 1 1 0.398028 1873 415

6 Light Blue 15th -1 1 0 0.626395 2702 135

Manual Rule ExtractionManual Rule Extraction

Automatic Rule Extraction: CCRE AlgorithmAutomatic Rule Extraction: CCRE Algorithm


The Fuzzy Vision algorithm employed in the game…

Currently, we still have not incorporated our new Fuzzy obstacle avoidance and Target seeking algorithm yet. Nonetheless, the robots are able to play soccer already.

Robots in Massey (IIMS Lab 7)


• We have presented a myriad of algorithms that allow colours to be enhanced or degraded through a hybrid fuzzy approach. Now colours can be processed by fuzzy colour contrast rules for more accurate colour object recognition. We have also incorporated rule extraction and colour learning algorithms for calibrating the colour contrast rules and target colour parameters automatically and more accurately.

ConclusionsConclusions

Introduction Colour Space Classification Fuzzy Contrast Fusion Rule Extraction ConclusionsLearning ConclusionsConclusions

Thanks for listening!

We are happy to answer any of your questions.


• Tested on 2 different cameras:– Samsung– AVT Marlin F-033C IEEE-1394 by Allied

Vision Technologies

• Tested on YUV, HSI and rg-Chromaticity colour spaces


**

We have presented an algorithm that We have presented an algorithm that resembles the colour discriminating feature of resembles the colour discriminating feature of the human visual system. Where our memory the human visual system. Where our memory serves as a guide in isolating an object’s serves as a guide in isolating an object’s colour from the scene, Fuzzy Colour contrast colour from the scene, Fuzzy Colour contrast rules are used to perform rules are used to perform colour correctionscolour corrections; ; thus, breaking the limits of accuracy of the pie thus, breaking the limits of accuracy of the pie slice-decision region for slice-decision region for colour object colour object identificationidentification. .


Results & AnalysisResults of Applying Colour Contrast Fusion in rg-chromaticity, YUV, and HSI Colour Spaces.

Light Blue Blue Violet

Colour Space

FP TP FP TP FP TP

Rg 0.0037 0.5908 0.0226 0.7139 0.0247 0.7786

Rg+ 0.0004 0.6183 0.0003 0.6023 0.0004 0.5957

YUV 0.0031 0.7324 0.0015 0.7844 0.0011 0.7907

YUV+ 0.0008 0.6473 0.0005 0.6821 0.0007 0.7447

HSI 0.0077 0.6149 0.1793 0.7984 0.0355 0.8288

HSI+ 0.0005 0.6183 0.0105 0.7733 0.0004 0.6746

**


Threshold = 0.5

Red HighRed High Red LowRed Low

Green HighGreen High Green LowGreen Low

Blue HighBlue High Blue LowBlue Low


The colour patches under bright illumination

Image taken by an overhead camera 10 feet from the ground

Experiments performed at IIMS Lab 7


After turning off some lights

Colours degrade considerably, reducing visibility of objects


Applying the colour contrast operations to compensate for the effects of glare, hue and saturation drifting allows for better colour spotting

Using the algorithm, Pink colours can be correctly classified


To some extent, the algorithm can see in the dark

Applying the colour contrast operations to compensate for the effects of glare, hue and saturation drifting also allows for colour correction


Documents

IntroductionColour SpaceClassificationFuzzy Contrast FusionRule ExtractionConclusionsLearningLearning D.Playne, V.Mehta, N.Reyes, A.Barczak Computer Science,