Upload
vuongque
View
219
Download
0
Embed Size (px)
Citation preview
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
Informatics luis rocha 2017
I501introduction
to informatics
introduction to informaticslecture 5
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Readings until now
This week Young, T. (2017). “Cease and desist”. Nature, 541(7637), 430–
430. Lazebnik, Y [2002]. "Can a biologist fix a radio?--Or, what I
learned while studying apoptosis". Cancer Cell, 2(3):179-182. Optional
Presentations G.L. Ciampaglia, P. Shiralkar, L.M. Rocha, J. Bollen, F. Menczer,
A. Flammini [2015]. “Computational fact checking from knowledge networks.” PLoS One. 10(6): e0128193. Russo, Giampaolo
Piantadosi, S. T.,et al (2011). Word lengths are optimized for efficient communication. PNAS, 108(9), 3526–3529. Malic, Vincent
Lecture Notes The Nature of Information Formalizing and Modeling the World
Available http://canvas.iu.edu and listed at
http://informatics.indiana.edu/rocha/academics/i501 Also check out
Links and notes at http://sciber.blogspot.com/
introduction to informatics
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
cybernetics
Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology Engineering-inspired Supremacy of mechanism
All can be axiomatized and computed Mculloch&Pitts’ work was major influence
“A logical calculus of the ideas immanent in nervous activity”. Bulletin of Mathematical Biophysics 5:115-133 (1943).
A Turing machine (any function) could be implemented with a network of simple binary switches (if circularity/feedback is present)
post-war science
Macy Conferences: 1946-53
Warren S. McCulloch
Claude ShannonMargaret Mead
Heinz Von Foerster
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
Informatics luis rocha 2017
I501introduction
to informatics
design principles of computationBabbage/Lovelace, Turing’s tape, and roles of information
distinction between numbers that mean things and numbers that do things.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
At every discrete time instance the machine is in a single state
Program is a state transition table
state Read symbol
Next state
Write symbol
Tape move
0 4 1 - left
1 - 0 1 right
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
Informatics luis rocha 2017
I501introduction
to informatics
Information as decrease in uncertainty . uncertainty-based information
C. E. Shannon, “A mathematical theory of communication”. Bell System Technical Journal, 27:379-423 and 623-656
Hartley, R.V.L., "Transmission of Information", Bell System Technical Journal, July 1928, p.535.
AAH 2log)( AAH 2log)(
Number of ChoicesMeasured in bits
i
n
iiS xpxpAH 2
1log)(
i
n
iiS xpxpAH 2
1log)(
Probability of alternativeMeasured in bits
information is surpriseinformation is surprise
including more structure reduces
surprise
including more structure reduces
surprise
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
uncertainty
Mutual Information Amount of information about one variable that can be gained
(uncertainty reduced) by observing another variable Information Gain (Kullback-Leibler Divergence)
Difference between two probability distributions p and q, average number of bits per data point needed in order to represent q
(model approximation) as it deviates from p (“true” or theoretical distribution)
Other measures
n
i
m
j ji
jiji ypxp
yxpyxpYXI
1 12
,log,);(
n
i
m
j ji
jiji ypxp
yxpyxpYXI
1 12
,log,);(
),()()();( YXHYHXHYXI ),()()();( YXHYHXHYXI
n
i i
ii xq
xpxpXqXpIG1
2log))(),((
n
i i
ii xq
xpxpXqXpIG1
2log))(),((
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
Informatics luis rocha 2017
I501introduction
to informatics
Do, do, do, do, do, do, do, do, doBefore we leave
Lemme tell y’all a lil’ somethingUptown Funk you up, Uptown Funk you up
Come on, danceJump on it
If you sexy, than flaunt itIf you freaky, than own it
Don’t brag about it, come show meCome on, dance
Jump on itIf you sexy, than flaunt it
Well, it’s Saturday night and we in the spot
Don’t believe me, just watchUptown Funk you up, Uptown Funk you up (say whaa?)
Uptown Funk you up, Uptown Funk you up
information of sequential messagesrate of removing uncertainty of each symbol
“syntactic” surprise But what about
function and meaning (semantics)?
“syntactic” surprise But what about
function and meaning (semantics)?
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Shannon’s mouse
trial and error algorithm information as reduction of uncertainty in the
presence of alternatives (combinatorics) lifelike behavior
trial and error to learn path from many alternatives adapts to new situations
how is learning achieved? Correct choices, information gained from reduced
uncertainty, must be stored in memory memory of information as a design principle
of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays
Brain as (switching) machine
controlling information to achieve life-like behavior
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
formalizing knowledge Lord Kelvin's dictum
“When you can measure what you are speaking of and express it in numbers you know that on which you are discoursing. But if you cannot measure it and express it in numbers. your knowledge is of a very meagre and unsatisfactory kind.” 1824-1907
Absolute scale of temperature, underwater telegraph cables, thermodynamics
Physics The first science to construct precise, rigorous
formal theories of the world. relating the operation of rules upon symbols to the
law-like behavior of the World. Aristotle (384-322 BC) was first to relate symbols
more explicitly to the external world and to successively clarify the nature of the symbol-world relation.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
understanding Nature with symbols Aristotle (384-322 BC)
First to relate symbols more explicitly to the external world and to successively clarify the nature of the symbol-world relation. Student of Plato, educated Alexander the Great first to consider specific observable factors which
determine motion. In Physics
he recognized (mathematical) rules which could describe the relation between an object's weight, the medium's density and the consequent rate of motion (fall): (1) for freely falling or freely rising bodies, speed is
proportional to the density of the medium. (2) in forced motion, speed is proportional to the force
applied and inversely proportional to the mass of the body moved
first time that observable quantities had been expressed in symbolic (numerical) form allowing the results of observations to be used in calculations The nature of causation http://classics.mit.edu/Aristotle/physics.html
Raphael’s “Plato and Aristotle”
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
abstracting relations Galileo (1564-1642)
Progressive dissociation of the symbols from objects The interrelationships among signs themselves
studied quite apart from the relations among the objects they represent Previously, symbols were still generally regarded as
inherent properties of the referent objects themselves Aristotle’s Physics postulated certain primary
qualities/elements such as “Fire”. Galileo regards "primary" properties as only those that can be mathematically quantified, such as size, shape and motion.
Newton (1643-1727) Extends process of abstraction
Distinguishes between symbols Arising from observation
represent initial conditions Arising from symbol relations
representing laws which govern the subsequent motion.
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Heinrich Hertz (1857-1894) Some facts about Hertz
First to broadcast and receive radio waves Established that light is a form of
electromagnetic radiation. His name is associated with the SI unit for
frequency Principles of Mechanics (1894)
Goal was to purge physics of mystical, undefined, unmeasured entities such as force (which one can infer but not
measure) Physical theories to be based only on
measurable quantities the results of measurements are symbols. Physical theory becomes about building
relationships among observationally-derived symbols: models what Hertz called "images."
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Using computation to model the World
World1
Measure
Symbols(Images)
Initial Conditions
Measure
Logical Consequence of Model
ModelFormal Rules
(syntax)
World2Physical Laws
Observed Result
Predicted Result????
Enco
ding
(Sem
antic
s)
(Pragmatics)
“The most direct and in a sense the most important problem which our conscious knowledge of nature should enable us to solve is the anticipation of future events, so that we may arrange our present affairs in accordance with such anticipation”. (Hertz, 1894)
Hertzian modeling paradigm
See: Modeling the World Lecture Notes
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
The Antikythera Mechanism 2,000-year-old astronomical calculator
bronze mechanical analog computer discovered more than 100 years ago in a Roman shipwreck, was used by
ancient Greeks to display astronomical cycles. built around the end of the second century BC to calculate
astronomical positions With imaging and high-resolution X-ray tomography to study how it
worked. complicated arrangement of at least 30 precision, hand-cut bronze gears
housed inside a wooden case covered in inscriptions. technically more complex than any known device for at least a millennium
afterwards.
Not a universal Turing machine, but an analog computer
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
let’s observe nature!
What do you see? Plants typically branch out How can we model that?
Observe the distinct parts Color them Assign symbols
Build Model Initial State: b b -> a a -> b a -> ab
Doesn’t quite Work!Psilophyta/Psilotum
bab
bb
bb
bb b
aa
aa
a aa
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Fibonacci Numbers!
Our First Model Rewriting production rules
Initial State: b b -> a a -> ab
n=0 : b n=1 : a n=2 : ab n=3 : aba n=4 : abaab n=5 : abaababa n=6 : abaababaabaab n=7 : abaababaabaababaababa
The length of the string is the Fibonacci Sequence 1 1 2 3 5 8 13 21 34 55 89 ...
Fibonacci numbers in Nature http://life-inspired.blogspot.pt/2015/02/links-for-modeling-
principles-of.html Romanesco: http://www.wussu.com/fractals/romanesco.htm
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
http://pithemovie.com
Mathematics Is The
Language Of Nature
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
what about our plant? An Accurate Model
Requires Varying angles Varying stem
lengths randomness
The Fibonacci Model is similar Initial State: b b -> a a -> ab
sneezewort
Psilophyta/Psilotum
bab
bb
bb
bb b
aa
aa
a aa
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
Assignment I
Due October 11th
Focus on uncovering quadrants using data
collection and induction.
Propose a formal model or algorithm of what each quadrant is doing. Analyze, using
deduction, the behavior of this algorithm.
Q1 Q2
Q3 Q4
The Black Box: Due October 11th, 2017
[email protected]/rocha/academics/i501/
INDIANAUNIVERSITY
I501introduction
to informatics
Informatics luis rocha 2017
next class
Week 6 Lecture
Klir, G.J. [2001]. Facets of Systems Science. Springer. Chapters: 1,2,3. Optional: Chapters 8, 11
Lecture Notes Formalizing and Modeling the World
Presentations & Discussion Gauvrit et al (2017). “Human behavioral complexity
peaks at age 25”. PLOS Computational Biology, 13(4), e1005408. Kresnye, Cassie
Schmälzle et al (2017). “Brain connectivity dynamics during social interaction reflect social network structure”. PNAS, 114(20), 5153–5158. Wong, Vincent
readings (available in Canvas and online)