Upload
felcia
View
41
Download
0
Tags:
Embed Size (px)
DESCRIPTION
CSC 4510 – Machine Learning. 7: Introduction to Neural Networks. Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www .csc.villanova.edu /~map/4510/. Some of the slides in this presentation are adapted from: - PowerPoint PPT Presentation
Citation preview
CSC 4510 – Machine LearningDr. Mary-Angela PapalaskariDepartment of Computing SciencesVillanova University
Course website: www.csc.villanova.edu/~map/4510/
7: Introduction to Neural Networks
1CSC 4510 - M.A. Papalaskari - Villanova University
Some of the slides in this presentation are adapted from:• Prof. Frank Klassner’s ML class at Villanova• the University of Manchester ML course http://www.cs.manchester.ac.uk/ugt/COMP24111/• The Stanford online ML course http://www.ml-class.org/
Machine learning problems• Supervised Learning
– Classification– Regression
• Unsupervised learning
Others: Reinforcement learning, recommender systems.
Also talk about: Practical advice for applying learning algorithms.
CSC 4510 - M.A. Papalaskari - Villanova University 2
Motivation – part 1• Learning a non-linear function
CSC 4510 - M.A. Papalaskari - Villanova University 3
You see this:
But the camera sees this:
What is this?
Computer Vision: Car detection
Testing:
What is this?
Not a carCars
Learning Algorithm
pixel 1
pixel 2
pixel 1
pixel 2
Raw image
Cars“Non”-Cars
pixel 1
pixel 2
Raw image
Cars“Non”-Cars
Learning Algorithm
pixel 1
pixel 2
pixel 1
pixel 2
Raw image
Cars“Non”-Cars
Learning Algorithm
pixel 1
pixel 2
Why not apply logistic regression?
Non-linear decision boundaries
x1
x2
Predict “ “ if
1-1
-1
1
Reminder: logistic regression can do non-linear
Add some higher-order terms?
pixel 1
pixel 2
Raw image
Cars“Non”-Cars
50 x 50 pixel images→ 2500 pixels (7500 if RGB)
pixel 1 intensity
pixel 2 intensity
pixel 2500 intensity
Quadratic features ( ): ≈3 million features
Learning Algorithm
pixel 1
pixel 2
Motivation – part 2• Take inspiration from the brain
CSC 4510 - M.A. Papalaskari - Villanova University 11
Neural Networks
Origins: Algorithms that try to mimic the brain.
• Was very widely used in 80s and early 90s;• popularity diminished in late 90s.• Recent resurgence: State-of-the-art technique for
many applications
Auditory cortex learns to see
Auditory Cortex
The “one learning algorithm” hypothesis
[Roe et al., 1992]
Somatosensory cortex learns to see
Somatosensory Cortex
The “one learning algorithm” hypothesis
[Metin & Frost, 1989]
Seeing with your tongue Human echolocation (sonar)
Haptic belt: Direction sense Implanting a 3rd eye
Sensor representations in the brain
[BrainPort; Welsh & Blasch, 1997; Nagel et al., 2005; Constantine-Paton & Law, 2009]
Neuron in the brain
Input signals sentfrom other neurons
If enough sufficient signals accumulate, the neuron fires
a signal.
Connection strengths determine how the signals are accumulated
Neurons in the brain
[Credit: US National Institutes of Health, National Institute on Aging]
Comparing Carbon & SiliconHuman Brain
Computational Units: O(1011) neuronsStorage Units: O(1011) neurons, O(1014) synapsesCycle Time: O(10-3) sec, Bandwidth: O(1014) bits/secNeuron Updates/sec: O(1014)
ComputerComputational Units: 1 CPU, 107 gatesStorage Units: O(1011) bits RAM, O(1012) bits diskCycle Time: O(10-8) sec, Bandwidth: O(108) bits/secNeuron Updates/sec: O(108)
The neuron modelMcCulloch & Pitts 1943
• Compute weighted sum of inputs and “fire” if above threshold value
CSC 4510 - M.A. Papalaskari - Villanova University 19
1x
2x
3x
add
)( taif 1output output
signal
• input signals ‘x’ and coefficients ‘w’ are multiplied
• weights correspond to connection strengths
• signals are added up – if they are enough, FIRE!
else0output
1w
2w
3w
i
M
iiwxa
1
incoming signal
connection strength
activationlevel
output signal
Activation Functions
A neuron can compute….
The perceptronRosenblatt 1958: Training algorithm Neural Net• A single neuron• Adjustable synaptic weights
CSC 4510 - M.A. Papalaskari - Villanova University 23
Perceptron learning1. Initialize weights and thresholds to random
numbers between -0.5 and 0.52. Activate perceptron3. Update weights:wi(p+1) = wi(p) + α * xi(p) * err(p)
4. Iterate until Convergence
Example: Perceptron
learning logical AND
25
Representation Limits for perceptron
Linear Separability