Embed Size (px)
Citation preview
Machine Learning for Physicists
Summer 2017University of Erlangen-NurembergFlorian Marquardt
(Image generated by a net with 20 hidden layers)
Lecture 1
INPUT
OUTPUT
(Picture: Wikimedia Commons)
INPUT
OUTPUT
(drawing by Ramon y Cajal,
~1900)
(Picture: Wikimedia Commons)
INPUT
OUTPUT
ArtificialNeural Network
input layer
output layer
INPUT
OUTPUT
input layer
output layer
“light bulb”
(this particular picture has never been seen before!)
(Picture: Wikimedia Commons)
input layer
output layer
“light bulb”
(training images)
(Picture: Wikimedia Commons)
1.2 million training pictures (annotated by humans)1000 object classes
ImageNet competition
2012: A deep neural network beats competition clearly (16% error rate; since then rapid decrease of error rate, down to about 7%)Picture: “ImageNet Large Scale Visual Recognition Challenge”, Russakovsky et al. 2014
Example applications of (deep) neural networks
Recognize imagesDescribe images in sentencesColorize imagesTranslate languages (French to Spanish, etc.)Answer questions about a brief textPlay video games & board games at superhuman level
(in physics:)predict properties of materialsclassify phases of matterrepresent quantum wave functions
(see links on website)e.g. http://machinelearningmastery.com/inspirational-applications-deep-learning/
Lectures Outline
• Basic structure of artificial neural networks• Training a network (backpropagation)• Exploiting translational invariance in image processing
(convolutional networks)• Unsupervised learning of essential features
(autoencoders)• Learning temporal data, e.g. sentences (recurrent
networks)• Learning a probability distribution (Boltzmann machine)• Learning from rare rewards (reinforcement learning)• Further tricks and concepts• Modern applications to physics and science
• Basic structure of artificial neural networks• Training a network (backpropagation)• Exploiting translational invariance in image processing
(convolutional networks)• Unsupervised learning of essential features
(autoencoders)• Learning temporal data, e.g. sentences (recurrent
networks)• Learning a probability distribution (Boltzmann machine)• Learning from rare rewards (reinforcement learning)• Further tricks and concepts• Modern applications to physics and science
Python
Lectures Outline
Learning by doing!
Keras package for
Python
Homework: (usually) explore via programming
We provide feedback if desired
No regular tutorial sessions
Homework
http://www.thp2.nat.uni-erlangen.de/index.php/2017_Machine_Learning_for_Physicists,_by_Florian_Marquardt
Homework
First homework:1.Install python & keras on your computer (see lecture homepage); questions will be resolved after second lecture THIS IS IMPORTANT!
2.Brainstorm: “Which problems could you address using neural networks?”
Next time: “installation party” after the lecture
Bring your laptop, if available (or ask questions)
8. 11.
22.29.
Mo Tu We Th Fr
May1.8.
19.26. 29.
Mo Tu We Th Fr
June
3. 6.10.17.24.
Mo Tu We Th
JulyMachine LearningLecture Schedule(Summer 2017)
Mo/Th 18-20
Very brief history of artificial neural networks
“Perceptrons”
“Backpropagation”
“Recurrent networks”
“Deep networks”become practical
Deep nets forimage recognition
beat the competition
A deep netreaches expert level in “Go”
50s/60s
80s (*1970)
80s/90s
early 2000s
2012
2015
“Convolutional networks”
1956 Dartmouth Workshop on Artificial Intelligence
Lots of tutorials/info on the web...
recommend: online book by Nielsen ("Neural Networks and Deep Learning") at https://neuralnetworksanddeeplearning.com
much more detailed book:“Deep Learning” by Goodfellow, Bengio, Courville; MIT press; see also http://www.deeplearningbook.org
Software –here: python & keras (builds on theano)
A neural network
= a nonlinear function (of many variables) that depends on many parameters
input layer
output layer
hiddenlayers
A neural network
y1 y2 y3input values
output of a neuron = nonlinear function of weighted sum of inputs
...
f(z)z =
X
j
wjyj + b
y1 y2 y3
w1 w2 w3weights
input values
output value
output of a neuron = nonlinear function of weighted sum of inputs
weighted sum
(offset, “bias”)
...
...
f(z)z =
X
j
wjyj + b
y1 y2 y3
w1 w2 w3weights
input values
output value
output of a neuron = nonlinear function of weighted sum of inputs
weighted sum
...
...
f(z)
sigmoid
reLU
z
(rectified linear unit)(offset, “bias”)
input layer
output layer
hiddenlayers
Each connection has a weight wEach neuron has an offset bEach neuron has a nonlinear function f (fixed)
The values of input layer neurons are fed into the network from the outside
A neural network
0.4 0.2 1.3 -10.1 0 00
0.4 0.2 1.3 -10.1 0 00
?
0.4 0.2 1.3 -10.1 0 00
0.5 0.1
?
0.4 0.2 1.3 -10.1 0 00
0.5
z=0.5*0.1+0.1*1.3
0.1
f(z)
0.4 0.2 1.3 -10.1 0 00
0.7
“feedforward” pass through the network: calculate output from input
0.4 0.2 1.3 -10.1 0 00
0.7-0.2 0.5 ...
“feedforward” pass through the network: calculate output from input
0.2
0.4 0.2 1.3 -10.1 0 00
0.7-0.2 0.5 ...
0.2 -2.3 ...
“feedforward” pass through the network: calculate output from input
0.5
0.2
0.4 0.2 1.3 -10.1 0 00
0.7-0.2 0.5 ...
0.2 -2.3
1.7 -0.3
...
“feedforward” pass through the network: calculate output from input
0.5
0.2
zj =X
k
wjkyink + bj
youtj = f(zj)
j=output neuronk=input neuron
input
output
z = wyin + b
in matrix/vector notation:
elementwise nonlinear function:
One layer
Python
u=dot(M,v)
for j in range(N): do_step(j)
x=linspace(-5,5,N)
def f(x): return(sin(x)/x)
Anaconda Jupyter
Python
u=dot(M,v)
for j in range(N): do_step(j)
x=linspace(-5,5,N)
def f(x): return(sin(x)/x)
Anaconda Jupyter
zj =X
k
wjkyink + bj
z = wyin + b
in matrix/vector notation:
z=dot(w,y)+b
A few lines of “python” !
Python code
matrix (Nout x Nin) vector (Nout)
vector (Nin)
vector (Nout)
Random weights and biases
Input values
Apply network!
N0=3
N1=2
A few lines of “python” !
input
output
