View
213
Download
0
Embed Size (px)
Citation preview
What I Did On My Summer Vacation: Undergraduate
Research Internships, Neural Networks, & Airport Security
J. McLean Sloughter
“Soon after the electrical current became known many attempts were made by the older physiologists to explain nervous impulses in terms of electricity. The analogy between the nerves of the body and a system of telephone or telegraph wires was too striking to be overlooked.”(from Studies in Advanced Physiology, Louis J. Rettger, A.M., 1898, p. 443)
2
An Extremely Over-Simplified Explanation The brain is made up of interconnected
neurons Neurons are binary – either fire or don’t fire As a neuron receives signals from other
neurons, it will start firing if the total signal reaches some threshhold
How
the
Bra
in W
orks
3
How
the
Bra
in W
orks
4
Just like that, except way more complicated Actually a lot more neurons involved Frequency of firing is also important
But let’s ignore those details for now…
How
the
Bra
in W
orks
5
Putting a philosophy degree to work
Warren McCulloch, a psychologist and philosopher, postulated that thought is discrete
Suggested a “psychon” – the smallest unit of thought
Thought that an individual neuron firing or not firing might be a psychon
Recommended developing a “calculus of ideas” to describe neural activity
His
tory
– 1
940s
6
Philosophy + Math = Fame
McCulloch teamed up with Walter Pitts, a math prodigy
Together they published “A Logical Calculus of the Ideas Immanent in Nervous Activity”
This paper introduced the idea of a “nervous network,” the first artificial neural model of cognitionH
isto
ry –
194
0s
7
Enter von Neumann
Von Neumann became an early proponent of their work
However, he criticized it as being overly simplistic
Based on some of von Neumann’s suggestions, McCulloch & Pitts proposed a system using a large number of neurons
This allows for robustness – an ability, for example, to recognize a slightly deformed square as still being essentially a square
His
tory
– 1
940s
8
Best Mathematician Name Ever
Norbert Weiner (“The Father of Cybernetics”) proposed a more involved system
Weighted inputs – one neuron can be more influential than another
Memory = learning weights Did not propose how this learning takes
place, dismissed that as a problem for engineers to deal with
His
tory
– 1
940s
9
In which not a whole lot happened
Marvin Minsky introduced a system based on behavioural conditioning
Neurons had probabilities of sending signals
When they produced the correct output, probabilities were increased
When the produced the wrong output, probabilities were decreased
And nobody really seemed to care (they were all busy becoming computer programmers)
His
tory
– 1
950s
10
Perceptrons
In 1960, Rosenblatt published a proof of the capabilities of what he named the “perceptron”
The perceptron acted much like the nervous network, but with weighted signals
The major advance was a learning algorithm Rosenblatt was able to prove that, using his
learning algorithm, any possible configuration of the perceptron could be learned, given the proper training data
His
tory
– 1
960s
11
Perceptron function
Consider a simple case where nodes A and B are each sending signals to node B
Node B has some threshold, T, which it needs to receive to be activated
A, B, and C are all binary – 0 or 1 W1 and W2 are the weights between A and C
and B and C Then, if A*W1 + B*W2 > T, C = 1 Otherwise, C = 0
His
tory
– 1
960s
12
Perceptron learning
Initialize weights randomly Set threshold to some arbitrary value (why does it not
matter what value the threshold is set to?) Randomly select one set of inputs Find the result based on current weights Subtract result from desired result = error term Look at each initial node individually
Multiply input value by error term by “learning coefficient” (between 0 and 1, controls amount of change you’ll allow at each iteration)
Add result to weight previously associated with that node to get a new weight
Pick a new set of inputs, repeat until convergence
His
tory
– 1
960s
13
Adaline
Widrow and Hoff created a system called Adaline – “Adaptive linear element”
Very similar to perceptrons (though with a slightly different learning algorithm)
Major changes were the use of -1 instead of 0 for no signal, and a “bias” term – a node that always fires
These were significant because they had no basis in neurophysiology, and were added purely because they could improve performance
His
tory
– 1
960s
14
The Wrath of Minksy
In 1969, Minsky again entered the world of neural networks, this time co-authoring the book “Perceptrons” with Seymour Papert
His
tory
– 1
960s
15
Xor
Minsky and Papert showed, among other critiques of perceptrons, that they weren’t capable of learning an exclusive OR (can you see why?)
An exclusive OR could be made by combining multiple other networks – have A and B feed into both an OR and a NAND, and then AND the results
But learning rules only worked with a single layer network – Minskey and Papert suggested researching whether learning rules could be developed for multi-layered networks
His
tory
– 1
960s
16
The Problem
Minsky & Papert put their critique of perceptrons at the front of the book
They put their suggestions for research into multi-layered perceptrons at the back of the book, after a few hundred pages of rather dense math
People didn’t seem to read that far Research on perceptrons died
His
tory
– 1
960s
17
Nothing important happenedH
isto
ry –
197
0s
18
The Multi-Layer Perceptron
Rumelhart, Hinton, and Williams created a learning algorithm for multi-layer perceptrons
Requires differentiation of functions, and thus the hard threshold had to be replaced by a sigmoid function
His
tory
– 1
980s
19
MLP function
Net input to a node:
Output from a node:
His
tory
– 1
980s
j
1
ijxw
n
j
I
eI
If
1
1)(
20
MLP learning
Change weight as follows:
Where is the learning coefficient, and E is the error term:
where
His
tory
– 1
980s
)(wij IEf
actualdesiredoutput yyE
n
j
outputj
imiddlei E
dI
IdfE
1
ijw)(
))(1)(()(
IfIfdI
Idf
21
The Problem
Metal detectors only detect things that are, well, metal (and even then only sometimes)
Lots of bad things aren’t metal – plastic explosives, ceramic guns, plastic flare guns
An x-ray could potentially see these objects, but submitting people to x-rays every time they fly isn’t an especially good ideaA
irpo
rt S
ecur
ity
22
The Solution
Scientists at Pacific Northwest National Laboratory developed a millimeter wave camera
Millimeter waves are not harmful like x-rays They can penetrate clothing, but are reflected
by skin Plastics and ceramics show up with a
distinctive speckled pattern, as they only partially reflect the waves
Air
port
Sec
urit
y
23
The New Problem Caused by the Solution Scientists at a
government lab just made a camera that can take pictures of you through your clothes
Implementing this in airports would have every passenger go through a virtual strip-search
Air
port
Sec
urit
y
24
The Solution to the Problem Caused by the Solution to the Other Problem Rather than have a human operator look at the
pictures, we can have a computer look at them for us The computer can identify suspicious areas and
provide a non-naughty picture to the security officer
Air
port
Sec
urit
y
25
In Practice
This technology is now in use by SafeView, a company spun off from this project
It is being used in airports, government buildings, border crossings, and other locations around the world
Air
port
Sec
urit
y
26
Student Research Opportunities I was involved in this project while a student intern at Pacific
Northwest National Lab Information about PNNL’s student internship programs can be
found online at http://science-ed.pnl.gov/students/ One of my summers on this project, I applied through the
Department of Energy’s internship program, which includes opportunities at a number of other national labs
Information on DOE internship programs is available at http://www.scied.science.doe.gov/scied/erulf/about.html
Res
earc
h In
tern
ship