DIGITAL IMAGE PROCESSING Dr J. Shanbehzadeh M. Hosseinajad ( J.Shanbehzadeh M. Hosseinajad)

DIGITAL IMAGE PROCESSING

Dr J. Shanbehzadeh M. Hosseinajad

( J.Shanbehzadeh M. Hosseinajad)

DIGITAL IMAGE PROCESSING

Dr J. [email protected]

M. Hosseinajad

Chapter 12 – Object RecognitionPart 2 – Neural Networks And Structural

Methods


mailto:[email protected]

Road map of chapter 8

12.1 12.312.1

12.1 Patterns and Pattern Classes12.2 Recognition Based on Decision-Theoretic Methods12.3 Structural Methods

Patterns and Pattern Classes

12.212.2 12.3

Some Basic Compression MethodsStructural Methods


12.1 Patterns and Pattern Classes

12.2 Recognition Based on Decision-Theoretic Methods

12.3 Structural Methods

Matching

Optimum Statistical Classifiers

Neural NetworksNeural Networks


Neural Networks




Try to mimic the structure and function of our nervous system.Neuron:

- A function unit of the nervous system.- Functioning through complex interconnection.- Functions in parallel.


Neural Networks





Biological Neurons: Each neuron receives thousands of signal from other neurons.If the integrated signal exceeds a threshold, the cell fires and generates an action potential or spike.

Artificial Neurons:A value of weight ‘w’ indicates the strength of the signal.Neuron B is stimulated if there is sufficient signal sent from neuron A.

Neural Networks

Mathematic Models12.1 Patterns and Pattern Classes




Neural Networks

Mathematic Models12.1 Patterns and Pattern Classes




Neural Networks





Perceptron is the simplest NN model consisting of a single neuron.Two main parameters:

WeightThreshold

Example: AND operation

Perceptron

Neural Networks





The rule which govern how exactly weights or thresholds are changed, is called as the learning algorithm.

Different types of neural networks may have different learning algorithms:

Fixed incremental correction ruleDelta ruleGradient decent rule…

Training

Neural Networks





Perceptron Learning Rule

Neural Networks






Neural Networks






Neural Networks






Combine both rulesw’ = w + α(r-o)x, w = α (r-o)x

where 0< α <1r = desired outputo = output from the network

Neural Networks





Fixed Increment CorrectionFor linearly separable classes:

The weight is updated when an error occurs.

case I:if y(k) w1 and w∈ T(k)y(k) ≤0 w(k+1) = w(k) + cy(k), c > 0

case II:if y(k) w2 and w∈ T(k)y(k) ≥0 w(k+1) = w(k) - cy(k), c > 0

case III:else w(k+1) = w(k)

Neural Networks





Nonseparable Classes

The Delta Rule

Can be used both with separable and non-separable classes.

Main Idea:– Minimizes the error between the actual and desired response at any training step.

– This can be achieved by using a technique called

Gradient descent.

Neural Networks





Gradient Descent

Gradient descent searches the space of weight vectors to find weights that best fit the training examples.

The objective is to minimize the following error:

E( w ) = ( ½ ) ∑( r – o )2o = wyr : desired outputo: output from the network

The training is a process of minimizing the error E( w ) in the direction of the steepest most rapid decrease, that is in direction opposite to the gradient.

Neural Networks





Gradient Descent

Neural Networks





Learning Rate

Neural Networks





How about XOR Problem?!

Two planes are required for correct classification.

Neural Networks





Multi-layer NN

Neural Networks





Learning Rule

Neural Networks





An Example

Neural Networks





Neural Networks





Neural Networks





Overfitting (overtraining): when the NN learns too many I/O examples it may end up memorizing the training data.

The size of the training set.

The architecture of the NN (too many hidden layers often causes overfitting problem).

The complexity of the problem at hand.

Since the algorithm attempts to minimize error, the algorithm can fall into any local minima.

Problems

Neural Networks





Number of Layers

The complexity of the decision surfaces can increase number of layers




Matching Shape Numbers

String Matching








Structural Pattern Recognition






The Order of Shape Number






The Order of Shape Number

Similarity Degree (k) between two region boundaries is defined as the largest order for which their shape numbers still coincide.

Distance between shape a and b = D(a,b)= 1/k





String MatchingString Matching






String Matching

Region a is coded into string denoted by a1 a2 ... an

Region b is coded into string denoted by b1 b2 … bn

The number of symbols that do not match is

A similarity b/w region a and b is the ratio R





String Matching

An Example

Documents

DIGITAL IMAGE PROCESSING Dr J. Shanbehzadeh M. Hosseinajad ( J.Shanbehzadeh M. Hosseinajad)