Upload
roz
View
62
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Physics and Machine Learning. “All the tricks that physicists’ use eventually end up in machine learning”. Energy – Physics definitions. - PowerPoint PPT Presentation
Citation preview
Physics and Machine Learning
“All the tricks that physicists’ use eventually end up in machine
learning”
Energy – Physics definitionsEnergy - A measure of being able to do work. There are many forms of energy, such as heat, mechanical, electrical, radiant, chemical, and nuclear energies. Energy is measured in such units as the joule (J), erg, kilowatt-hour (kW-hr), kilocalorie (kcal), foot-pound (ft-lb.), electron-volt (ev), and British thermal unit (BTU). –NASA.gov
"It is important to realize that in physics today, we have no knowledge of what energy is. We do not have a picture that energy comes in little blobs of a definite amount." -Richard Feynman "Lectures on Physics"
Energy - Physics II
Images: http://meso.phys.northwestern.edu/research/magneticarrays.html
• Magnetic Arrays and Spin
Energy - Machine Learning
• H=-½iJwiJSiSJ Si SJWiJ
Vs.
Vs.
Vs. Vs.
Energy - Machine Learning II
• Energy is the difference in weight between all nodes that agree and all nodes that disagree.
• The more weights, the greater energy. • The “closer” the call, the lower |H|.
Energy Minima
• Retrieval States – attractors
• Mixture States – linear combinations of odd numbered attractors
• Spin Glass States – uncorrelated to attractors.
Ferromagnetics
Energy MetaphorImagine the atomic magnets as movable objects with the freedom to flip, but you control their position. Each iteration of learning is like forcing all magnets to be closer together, as such the network energy is potential energy and the flipping of spins is the expression of kinetic energy.
Temperature – Physics I
• Extending the ferromagnetic example• As temperature increases, the impact of
other atomic magnets’ spins is decreased.• At absolute zero, temperature has no
impact.• At the critical temperature(Tc), spin has no
impact.
Temperature – Ferromagnetics
• Si = +1 w/ probability g(hi); else -1• g(h) = 1/(1+exp(-2βh))• β = 1/(kBT)
• kB = Boltzman’s constant• T = temperature• Fβ(+/-hi)=1/(1+exp(-/+ 2βhi) Fβ(+/-hi)• Fβ(+/-hi) is a logistic function
Temperature – Machine Learning
• Logistic function• Noise• Used in the elimination of spurious
local minima
Mean Field Theory – Physics• The individual measurement and
summation of each member of a magnetic array is too expensive
• Physicists look to average values as an inexpensive way to extract further truth from a complex combinatronics problem.
Mean Field Theory – Physics• hi=JwiJSJ+hext
• <hi>=JwiJ<SJ>+hext
• <Si>=tan(β<hi>) = tanh(βJwiJ<SJ>+hext)• <S>=tanh(βJ(S))
<S>
TcT
1
Mean Field Theory – stochastic model
• <Si>=tanh(β/NJuζuiζu
J<SJ>)• We allow an assumption, that <Si> is
proportional to one of the stored patterns ζv
i • <Si>=m ζv
i
• <Ncorrect>=½N(1+m)
TcT
Mean Field Theory • <Ncorrect>=½N(1+m)• There is a point at which noise
overcomes the ability of a network to make an informed decision.
Conclusions• All these metaphors that pull from
physics are very tightly linked to energy.
• All metaphors concentrate on atomic events. (exception that proves the rule: mean field theory).
Extra Time? Extra Topics!
Entropy – Physics• The inevitable progression toward
chaos• The motion of energy and matter
away from an organized state.
Entropy – Machine Learning• S = -PlogP
• For S = -Plog2P (the binary case) this is the average amount of additional information required to specify one of the staties.
Quantum MechanicsFits within the context of our
expectation for where to look for Physics crossover
• Atomic – discrete and binary• Energy specificLast Class Lecture – use of Dyads.