Ch3 Ann Fs Presentation

8/9/2019 Ch3 Ann Fs Presentation

1/32

28/01/15 1

Perceptron Perceptron is one of the earliest models of articial neuron.

It was proposed by osenblatt in 1!58.

It is a sin"le layer neural networ# whose wei"hts can be

trained to produce a correct tar"et $ector when presentedwith the correspondin" input $ector

%he trainin" techni&ue used is called the Perceptronlearning rule.

%he Perceptron "enerated "reat interest due to its ability to

generalizefrom its trainin" $ectors and wor# withrandomly distributed connections.

Perceptron are especially suited for problems in patternclassication.


2/32

28/01/15 2

Perceptrons

'inear separability

( set of )2*+ patterns )x1,x2+ of two classes is linearly

separable if there e-ists a line on the )x1,x2+ plane

w0 w1x1 w2x2 0

eparates all patterns of one class from the otherclass

( perceptron can be built with

inputx0 1,x1,x2with wei"hts w0, w1, w2

ndimensional patterns )x1,,xn+ 3yperplane w0 w1x1 w2x2 wnxn 0

di$idin" the space into two re"ions 4an we "et the wei"hts from a set of sample patterns

If the problem is linearly separable, then 67 )byperceptron learnin"+


3/32

28/01/15

LINEAR SEPARABILITY Definition:Two sets of points A and B in an n-dimensional space are

called linearly separable if n+1 real numbers w1, w2, w3, . . . ., wn+1exist,

such that eery point !x1, x2, . . . , xn"A satisfies and eery point !x1, x2,. . . , xn" B satisfies .

Absolute #inear $eparability Two sets of points A and B in an n-dimensional space are called linearly

separable if n+1 real numbers w1, w2, w3, . . . ., wn+1exist, such thateery point !x1, x2, . . . , xn" A satisfies and eery point !x1, x2, . . . , xn"B satisfies .

%wo nite sets of points ( and , in n9dimensional space whichare linear separable are also absolute linearly separable.

In "eneral, absolute linearly separable9: linearly separable

but if sets are nite, linearly separable absolutely linearlyseparable


4/32

7-amples of linearly separableclasses

9 'o"ical AND function

patterns )bipolar+ decisionboundary

-1 -2 output w1 191 91 91 w2 1

91 1 91 w0 911 91 911 1 1 91 -1 -2 0

9 'o"ical OR function

patterns )bipolar+ decisionboundary

-1 -2 output w1 191 91 91 w2 1

91 1 1 w0 11 91 11 1 1 1 -1 -2 0

x

oo

o

x: class I (output = 1)o: class II (output = -1)

x

xo

x

x: class I (output = 1)o: class II (output = -1)

28/01/15 ;


5/32

28/01/15 5

Perceptron


6/32

28/01/15 ?

in"le 'ayer *iscrete Perceptron @etwor#s

)'*P+

&lass&1

&lass &2

x1

x2

i!. ".2 Illustration of t$e $'per plane (in t$is example a strai!$t line)

as decision oundar' for a two dimensional two-class patron classification prolem.

%o de$elop insi"ht into the beha$ior of a pattern classier, it isnecessary to plot a map of the decision re"ions in n9dimensionalspace, spanned by the n input $ariables. %he two decision re"ionsseparated by a hyper plane dened by

=

=n

i

iw

0

i0x


7/32

28/01/15 A

'*P

&la

ss

&1

&la

ss

&2

()

&la

ss

&1

&la

ss

&2

(a)

*ecision oundar'

i! (a) + pair of linearl' separale patterns

() + pair of nonlinearl' separale patterns.

Bor the Perceptron to function properly, the two classes 41 and 42 must belinearly separable.

In Bi"..)a+, the two classes 41 and 42 are suCciently separated fromeach other to draw a hyper plane )in this it is a strai"ht line+ as thedecision boundary.


8/32

28/01/15 8

'*P


9/32

28/01/15 !

'*P


10/32

28/01/15 10

*iscrete Perceptron trainin"

al"orithm


11/32

28/01/15 11

(l"orithm continued..


12/32

28/01/15 12

(l"orithm continued..


13/32

28/01/15 1

7-ample=

uild the Perceptron networ# to realiDe fundamental lo"ic"ates, such as (@*, E and FE.


14/32

28/01/15 1;


15/32

28/01/15 15


16/32

28/01/15 1?


17/32

28/01/15 1A


18/32

1 2 3 4 5 6 7 8 9 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of epochs

Error

1 2 3 4 5 6 7 8 9 100

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Number of epochs

Error

Bi". .5 %he 7rror prole durin" thetrainin" of Perceptron to learn input9

output relation of (@* "ate

Bi". .; %he 7rror prole durin" the trainin"

of Perceptron to learn input9output relation ofE "ate

esults

28/01/15 18


19/32

0 5 10 15 20 25 30 35 40 45 500.5

1

1.5

2

2.5

Number of epochs

Error

Bi". .? %he 7rror prole durin" the trainin" ofPerceptron to learn input9output relation of FE"ate

28/01/15 1!


20/32

28/01/15 20

Single-Layer Continuous Perceptronnetwors)'4P+

%he acti$ation function that is used in modelin" the4ontinuous Perceptron is si"moidal, which isdi>erentiable.

%he two ad$anta"es of usin" continuous acti$ationfunction are )i+ ner control o$er the trainin" procedureand )ii+ di>erential characteristics of the acti$ationfunction, which is used for computation of the error"radient.

%his "i$es the scope to use the "radients in modifyin"

the wei"hts. %he "radient or steepest descent method isused in updatin" wei"hts startin" from any arbitrarywei"ht $ector G, the "radient 7)G+ of the current errorfunction is computed.


21/32

28/01/15 21

Single-Layer Continuous Perceptronnetwors


22/32

28/01/15 22

'4P


23/32

28/01/15 2

'4P


24/32

28/01/15 2;

'4P


25/32

28/01/15 25

'4P


26/32

28/01/15 2?

Perceptron Con!ergenceT"eore#


27/32

28/01/15 2A



28/32

28/01/15 28



29/32


30/32

28/01/15 0



31/32

28/01/15 1


32/32

28/01/15 2

'imitations of Perceptron

%here are limitations to the capabilities of Perceptronhowe$er.

%hey will learn the solution, if there is a solution to be found. Birst, the output $alues of a Perceptron can ta#e on only

one of two $alues )%rue or Balse+. econd, Perceptron can only classify linearly separablesets

of $ectors. If a strai"ht line or plane can be drawn toseparate the input $ectors into their correct cate"ories, theinput $ectors are linearly separable and the Perceptron willnd the solution.

If the $ectors are not linearly separable learnin" will ne$erreach a point where all $ectors are classied properly.

%he most famous e-ample of the PerceptionHs inability tosol$e problems with linearly non9separable $ectors is theboolean FE realiDation.

Documents

Ch3 Ann Fs Presentation