View
411
Download
0
Category
Tags:
Preview:
Citation preview
Lecture Content
• Introduction and a historical review:
– History of neurocomputing.
– Overview of neurocomputing.
2Neural Networks Dr. Randa Elanwar
History of Neurocomputing
• Alexander Bain (1873) claimed that both thoughts and body activity resulted from interactions among neurons within the brain.
• For Bain, every activity led to the firing of a certain set of neurons. When activities were repeated, the connections between those neurons strengthened. According to his theory, this repetition was what led to the formation of memory.
• The general scientific community at the time was doubting Bain’s theory because it required what appeared to be an excessively large number of neural connections within the brain.
3Neural Networks Dr. Randa Elanwar
History of Neurocomputing
• It is now apparent that the brain is exceedingly complex and that the same brain “wiring” can handle multiple problems and inputs.
• Neural network theory has served both to:
– better identify how the neurons in the brain function and
– provide the basis for efforts to create artificial intelligence.
4Neural Networks Dr. Randa Elanwar
Overview of Neurocomputing
• The brain is said to be composed of natural neural network, I.e., mass of highly non-linear parallel inter-connected computational units called neurons.
• Each neuron is connected to many other neurons
• Neurons transmit signals to each other
• Whether a signal is sent, depends on the strength of the bond between two neurons
• Whether a signal is transmitted is an all-or-nothing event (the electrical potential in the cell body of the neuron is thresholded: fired up/ activated)
5Neural Networks Dr. Randa Elanwar
Brain vs. Digital Computers
- Computers require hundreds of cycles to simulate a firing of a neuron.
- The brain can fire all the neurons in a single step. Parallelism
- Serial computers require billions of cycles to perform some tasks but the brain takes less than a second.
e.g. Face Recognition
6Neural Networks Dr. Randa Elanwar
Overview of Neurocomputing
Neural Networks Dr. Randa Elanwar 8
Overview of Neurocomputing
What are Neural Networks
• Neural Networks (NNs) are networks of neurons, for example, as found in real (i.e. biological) brains.
• Artificial Neurons are crude approximations of the neurons found in brains. They may be physical devices, or purely mathematical constructs.
• Artificial Neural Networks (ANNs) are networks of Artificial Neurons, and hence constitute crude approximations to parts of real brains. They may be physical devices, or simulated on conventional computers.
Neural Networks Dr. Randa Elanwar 9
Overview of Neurocomputing
What are Neural Networks
• From a practical point of view, an ANN is just a parallel computational system, consisting of many simple processing elements, connected together in a specific way in order to perform a particular task, which is difficult to traditional (serial) computers.
• One should never lose sight of how crude the approximations are, and how over-simplified our ANNs are compared to real brains.
Comparison between brain verses ANN
Neural Networks Dr. Randa Elanwar 10
Brain ANN
Speed Few ms. Few nano sec. massive parallel
processing
Size and
complexity
1011 neurons & 1015
interconnections
Depends on designer
Storage
capacity
Stores information in its
interconnection (synapse)
No Loss of memory
Contiguous memory locations
loss of memory may happen
sometimes.
Tolerance Has fault tolerance Limited fault tolerance:
Information gets disrupted when
interconnections are disconnected
Control
mechanism
Complicated involves
chemicals in biological neuron
Simpler in ANN
Neural Networks Dr. Randa Elanwar 11
NON-LINEARITYIt can model non-linear systems
INPUT-OUTPUT MAPPINGIt can derive a relationship between a set of input & output responses
ADAPTIVITYThe ability to learn allows the network to adapt to changes in the surrounding environment
EVIDENTIAL RESPONSEIt can provide a confidence level to a given solution
Advantages of NN
Neural Networks Dr. Randa Elanwar 12
CONTEXTUAL INFORMATIONKnowledge is presented by the structure of thenetwork. Every neuron in the network ispotentially affected by the global activity of allother neurons in the network. Consequently,contextual information is dealt with naturally inthe network.
FAULT TOLERANCEDistributed nature of the NN gives it fault tolerant capabilities
NEUROBIOLOGY ANALOGYModels the architecture of the brain
Advantages of NN
Neural Networks Dr. Randa Elanwar 15
How does the nervous system works?
• The human nervous system can be broken down into three stages that may be represented in block diagram form as:– The receptors collect information from the environment – e.g. photons on the retina.– The effectors generate interactions with the environment – e.g. activate muscles.– The flow of information/activation is represented by arrows – feed forward and
feedback.
Overview of Neurocomputing
Overview of Neurocomputing
The structure of a biological neuron
Neural Networks Dr. Randa Elanwar 16
•A biological neuron has three types of main components; dendrites, soma (or cell body) and axon.
•Dendrites receives signals from other neurons.
•The soma, sums the incoming signals. When sufficient input is received, the cell fires; that is it transmit a signal over its axon to other cells.
17
Neural network: Definition
• Neural network: information processing paradigm inspiredby biological nervous systems, such as our brain
• Structure: large number of highly interconnectedprocessing elements (neurons) working together
• Like people, they learn from experience (by example)
Neural Networks Dr. Randa Elanwar
Artificial Neural Network: Definition
• The idea of ANN: NNs learn relationship between cause and effect or organize large volumes of data into orderly and informative patterns.
• Definition of ANN: “Data processing system consisting of a large number of simple, highly interconnected processing elements (artificial neurons) in an architecture inspired by the structure of the cerebral cortex of the brain”
(Tsoukalas & Uhrig, 1997).
18Neural Networks Dr. Randa Elanwar
Why Study Artificial Neural Networks?
• Artificial Neural Networks are powerful computational systems consisting of many simple processing elements connected together to perform tasks analogously to biological brains.
• They are massively parallel, which makes them efficient, robust, fault tolerant and noise tolerant.
• They can learn from training data and generalize to new situations.
• They are useful for brain modeling and real world applications involving pattern recognition, function approximation, prediction, …
19Neural Networks Dr. Randa Elanwar
What are Artificial Neural Networks Used for?
• Brain modeling– Models of human development – help children with developmental problems– Simulations of adult performance – aid our understanding of how the brain
works– Neuropsychological models – suggest remedial actions for brain damaged
patients
• Real world applications– Financial modeling – predicting stocks, shares, currency exchange rates– Other time series prediction – climate, weather, airline marketing tactician– Computer games – intelligent agents, backgammon, first person shooters– Control systems – autonomous adaptable robots, microwave controllers– Pattern recognition – speech recognition, hand-writing recognition, sonar
signals– Data analysis – data compression, data mining– Noise reduction – function approximation, ECG noise reduction– Bioinformatics – protein secondary structure, DNA sequencing
20Neural Networks Dr. Randa Elanwar
Recommended