S. Mandayam/ ANN/ECE Dept./Rowan University Smart Sensors 0909.504.01/0909.402.01 Spring 2004 Shreekanth Mandayam ECE Department Rowan University Artificial

S. Mandayam/ ANN/ECE Dept./Rowan University

Smart SensorsSmart Sensors0909.504.01/0909.402.010909.504.01/0909.402.01

Spring 2004Spring 2004

Shreekanth MandayamECE Department

Rowan University

Artificial Neural NetworksArtificial Neural NetworksLecture 1Lecture 1

March 1, 2004March 1, 2004


PlanPlan

• What is artificial intelligence?• Course module objectives• Historical development – the neuron

model• The artificial neural network paradigm• What is knowledge? What is learning?• The Perceptron• The “Future”….?


Artificial IntelligenceArtificial Intelligence

Systems that think like humans• Cognitive modeling

Systems that think rationally• Logic

Systems that act like humans• Natural language processing• Knowledge representation• Machine learning

Systems that act rationally• Decision theoretic agents


ObjectivesObjectives

• At the conclusion of this course module the student will be able to:• Identify and describe engineering

paradigms for knowledge and learning• Identify, describe and design artificial

neural network architectures for simple cognitive tasks


Biological OriginsBiological Origins


Biological OriginsBiological Origins


History/PeopleHistory/People

1940’s Turing General problem solver, “Turing test”

1940’s Shannon Information theory

1943 McCulloch and Pitts Math of neural processes

1949 Hebb Learning model

1959 Rosenblatt The “Perceptron”

1960 Widrow LMS training algorithm

1969 Minsky and Papert Perceptron deficiency

1985 Rumelhart Feedforward MLP, backprop

1988 Broomhead and Lowe Radial basis function neural nets

1990’s VLSI implementations


Neural Network ParadigmNeural Network ParadigmStage 1: Network Training

ArtificialArtificialNeuralNeural

NetworkNetwork

Present Examples Indicate Desired Outputs

DetermineSynapticWeights


NetworkNetworkNew Data Predicted Outputs

Stage 2: Network Testing

“knowledge”


ANN ModelANN Model


NetworkNetwork

xInput

Vector

yOutputVector

fComplexNonlinearFunction

3

2

1

x

x

x

3

2

1

y

y

y

f(x) = y

“knowledge”


Popular I/O MappingsPopular I/O Mappings

ANNx y

Single output

y1

ANNx

1-out-of-c selector

y2

yc

y1

ANNx

Coder

y2

yc

ANNx

Associator

y


The PerceptronThe Perceptron

(.)

wk1

wk2

wkm

x1

x2

xm

Inpu

ts

Synapticweights

Bias,bk

Induced field,

vk

Output,ykuk

Activation/ squashing function


““Learning”Learning”

[w]x y

ANN

Mathematical Model of the Learning Process

[w]0x y(0)

Intitialize: Iteration (0)

[w]1x y(1)

Iteration (1)

[w]nx y(n) = d

Iteration (n)desiredo/p


Learning RulesLearning Rules

• Error Correction Learning• Delta Rule or Widrow-Hoff Rule

• Memory Based Learning• Nearest Neighbor Rule

• Hebbian Learning

• Competitive Learning

• Boltzman Learning


Error-Correction LearningError-Correction Learning

(.)

wk1(n)x1 (n)

x2

xm

Inpu

ts

Synapticweights

Bias,bk

Induced field,vk(n)

Activation/ squashing function

wk2(n)

wkm(n)

Output,yk (n)

Desired Output,dk (n)

ErrorSignalek (n)

+

-


Learning ParadigmsLearning Paradigms

Environment(Data)

Teacher(Expert)

ANN

error

desired

actual

+-

Supervised Unsupervised


Learning ParadigmsLearning Paradigms

Supervised Unsupervised

Environment(Data)

Delay

ANN

Delayed ReinforcementLearning

CostFunction


Learning TasksLearning Tasks• Pattern Association

• Pattern Recognition

• Function Approximation

• Filtering

ClassificationClassification

x1

x2

1

2

DB

x1

x2

1

2

DB


Perceptron Training Perceptron Training Widrow-Hoff Rule (LMS Algorithm)Widrow-Hoff Rule (LMS Algorithm)

w(0) = 0

n = 0

y(n) = sgn [wT(n) x(n)]

w(n+1) = w(n) + [d(n) – y(n)]x(n)

n = n+1

Matlab Demo


The Age of Spiritual MachinesWhen Computers Exceed Human Intelligenceby Ray Kurzweil | Penguin paperback | 0-14-028202-5 |


SummarySummary

Documents

S. Mandayam/ ANN/ECE Dept./Rowan University Smart Sensors 0909.504.01/0909.402.01 Spring 2004 Shreekanth Mandayam ECE Department Rowan University Artificial