Introduction to Neural Networks

Presented by:

Hafiz Syed Adnan Ahmed

Introduction

• Artificial Neural Network is based on the biological nervous

system as Brain

• It is composed of interconnected computing units called

neurons

• ANN like human, learn by examples

Why Artificial Neural Networks?There are two basic reasons why we are interested in building artificial neural networks (ANNs):

• Technical viewpoint: Some problems such as character recognition or the prediction of future states of a system require massively parallel and adaptive processing.

• Biological viewpoint: ANNs can be used to replicate and simulate components of the human (or animal) brain, thereby giving us insight into natural information processing. 3

Science: Model how biological neural systems, like human brain, work?

• How do we see?• How is information stored in/retrieved

from memory?• How do you learn to not to touch fire?• How do your eyes adapt to the

amount of light in the environment?• Related fields: Neuroscience,

Computational Neuroscience, Psychology, Psychophysiology, Cognitive Science, Medicine, Math, Physics. 4

Real Neural Learning

• Synapses change size and strength with experience.

• Hebbian learning: When two connected neurons are firing at the same time, the strength of the synapse between them increases.

• “Neurons that fire together, wire together.”

5

Biological Neurons• Human brain = tens of thousands

of neurons• Each neuron is connected to

thousands other neurons• A neuron is made of:

• The soma: body of the neuron• Dendrites: filaments that provide

input to the neuron• The axon: sends an output signal• Synapses: connection with other

neurons – releases certain quantities of chemicals called neurotransmitters to other neurons

6

Modeling of Brain Functions

7

Modelling a Neuron

• aj :Activation value of unit j

• wj,I :Weight on the link from unit j to unit i

• inI :Weighted sum of inputs to unit i

• aI :Activation value of unit i• g :Activation function

j

jiji aWin ,

What is an artificial neuron ?

• Definition : Non linear, parameterized function with restricted output range

1

10

n

iiixwwfy

x1 x2 x3

w0

y

Simple Neuron

X1

X2

Xn

OutputInputs

b

An Artificial Neuron

x1

x2

xn

…

Wi,1

Wi,2

…

Wi,n

n

jjjii txtwt

1, )()()(net

xi

neuron i

net input signal

synapses

output ))(()(x tnetft iii

Activation functions

0 2 4 6 8 10 12 14 16 18 200

2

4

6

8

10

12

14

16

18

20

xy

-10 -8 -6 -4 -2 0 2 4 6 8 10-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

-10 -8 -6 -4 -2 0 2 4 6 8 10-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

Linear

Logistic

Hyperbolic tangent

)exp(1

1

xy

)exp()exp(

)exp()exp(

xx

xxy

How do NNs and ANNs work?• Information is transmitted as a series of

electric impulses, so-called spikes.

• The frequency and phase of these spikes encodes the information.

• In biological systems, one neuron can be connected to as many as 10,000 other neurons.

• Usually, a neuron receives its information from other neurons in a confined area

13

Navigation of a car

• Done by Pomerlau. The network takes inputs from a 34X36 video image and a 7X36 range finder. Output units represent “drive straight”, “turn left” or “turn right”. After training about 40 times on 1200 road images, the car drove around CMU campus at 5 km/h (using a small workstation on the car). This was almost twice the speed of any other non-NN algorithm at the time.

14

15

Automated driving at 70 mph on a public highway

Camera image

30x32 pixelsas inputs

30 outputsfor steering

30x32 weightsinto one out offour hiddenunit

4 hiddenunits

Computers vs. Neural Networks

“Standard” Computers Neural Networks

one CPU highly parallelprocessing

fast processing units slow processing units

reliable units unreliable units

static infrastructure dynamic infrastructure

16

Neural Network

Input Layer Hidden 1 Hidden 2 Output Layer

Network Layers

The common type of ANN consists of three layers

of neurons: a layer of input neurons connected to

the layer of hidden neuron which is connected to

a layer of output neurons.

Architecture of ANN

• Feed-Forward networksAllow the signals to travel one way from input to

output• Feed-Back NetworksThe signals travel as loops in the network, the

output is connected to the input of the network

Comparison of Brains and Traditional Computers

• 200 billion neurons, 32 trillion synapses

• Element size: 10-6 m

• Energy use: 25W• Processing speed: 100 Hz• Parallel, Distributed• Fault Tolerant• Learns: Yes• Intelligent/Conscious:

Usually

• 1 billion bytes RAM but trillions of bytes on disk

• Element size: 10-9 m• Energy watt: 30-90W (CPU)• Processing speed: 109 Hz• Serial, Centralized• Generally not Fault Tolerant• Learns: Some• Intelligent/Conscious:

Generally No

Neural Networks (Applications)

• Face recognition• Time series prediction• Process identification• Process control• Optical character recognition• Adaptative filtering• Etc…

And Finally….

“If the brain were so simple that we could understand it then we’d be so simple that

we couldn’t”

Introduction is End of Neural Networks