28

[email protected]. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Embed Size (px)

Citation preview

Page 2: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Meaning is the computational exploitation of the underlying structure of the world, and mind is

execution of an evolved program, mostly encoded in the DNA, that is all about meaning.

The program can only be so compact and so effective by code reuse, developing modules corresponding to real concepts in the world,

reused in metaphor and pleiotropy.

Page 3: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

The mind is a computer program

• Effective procedure = sequence of syntactic steps• Church-Turing Thesis: any physical system

simulatable by Turing machine

Page 4: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Problems (Searle, Dreyfus,Chalmers…):

• How can my feeling of awareness etc come from syntax?

• How can syntax give rise to semantics?

• How can syntax correspond to events in the world?

• Why can’t computers understand?

Page 5: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Occam’s Razor

• Find a sufficiently compact description of enough data, and it corresponds to the world.

• Simplest example: curve fitting

Page 6: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

3 Formal Views of Occam’s Razor

• Vapnik-Chervonenkis Dimension

• Minimum Description Length

• Bayesian Probability

Compactness sufficient, more or less necessary, for generalization.

Need inductive bias to learn.

Page 7: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Extrapolated Occam

• A compact enough program that behaves effectively after enough reinforcement learning trials exploits underlying structure and will continue to behave effectively.

• It is compact by virtue of code reuse, developing modules corresponding to real concepts in the world, reused in metaphor and pleiotropy.

Page 8: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Why computers don’t understand

• AI programs not compact.

List of answers not compact

Expert systems not compact, because:

Human programmers not capable of compaction.• Academic field of AI splits world into vision, planning,

learning, natural language,etc, throwing out structure

before it starts.“The girl saw the dog in the window. She wanted it.”

Page 9: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Exploiting structure

• Is 7788779909777885687654 even?

• Extracting compact structure and exploiting it separate computationally hard problems.

• Discuss chess, Go, Blocks World, Rubik’s cube, traveling salesman… and real world.

• Show evolution of Hayek Machine to solve problems, model of insight.

Page 10: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Exploiting Structure

• AI approach: brute force search• Human approach: build compact code using preexisting

relevant modules• Computer Science approach: learn new module, for

recursion.• Hayek: evolves program solving arbitrary instances.

Page 11: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Hayek Machine ApproachRun artificial economy.Agents are computer programs.Insert new computer programs.

$

$$

$ $

$

$

$

$

$$

•Everything owned.•Conservation of money.

•If agents rational, new agent can enter if and only if improves performance of the system•Hope for evolution to rationality.

Page 12: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Cleaner Cleaner Stacker 438Stacker 46 Stacker 19 Closer

Done

Strategy

Agents bid roughly A · NumCorrect + B

Cleaner(s), Stackers, Closer.

19

Page 13: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Are there objects in the world?

• Why call a collection of atoms never before seen a cup?

• In what sense is a newspaper one object?• Do electrons exist, even if you’ve never seen one?• Does the platonic ideal of a circle exist?

Page 14: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Answer: mind code exploits structure in world

• Mind consists of modular program, with modules that exploit compact structure of world, achieving compactness through code reuse.

• Modularity from psychophysics, neuroanatomy, stroke victims, imaging,…

• Metaphor = code reuse. Metaphor pervasive eg time is money (Lakoff and Johnson)-- buy,spend,borrow,waste etc time.

Page 15: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Exploiting Structure is NP-Hard

• Example: Graph 3-coloring

Page 16: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Constraint Propagation (Blum)

Page 17: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Constraint Propagation

Page 18: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Constraint Propagation

Page 19: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Constraint Propagation

Page 20: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Constraint Propagation

Page 21: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Compact Explanation is DNA

• 1-10 MB, a fraction of Powerpoint source code. Brain 100 million times larger.

• 10**35 creatures/RL trials, little computation in learning during life (LDL).

• LDL highly biased in– many examples, including grammar learning in humans.

• Evolution of learning to be expected, learning and development two faces of coin.

Page 22: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Evolution of Learning

We are born stupid-- but learn?• Answer: yes, but the crux is in the DNA. Inductive biasDNA programs development interacting with

world-- learning is just sophisticated development, largely predestined. Ferret , critical periods, …

Grammar is an excellent example.Once constrained to meaningful quantities only,

built on top of programmed modules, learning is fast, almost automatic.

Page 23: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Evolution of Language

• Computational modules present in animals

• Words are labels for modules

• Solves Gavagai problem.

• Language evolution stuck in local optima till humans (Nowak).

• Language allows cumulative progress in building program.

Page 24: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Why people are different than apes.

• Grow huge modular program

• Because of language, progress cumulative

• Hard search problem, finding program… but we have invested cumulative effort comparable to that evolution invested in chimps.

• Affects everything, eg. Theory of mind.

Page 25: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

Evolution of Consciousness• Evolve: “sovereign agents” that maximize inborn rewards.

• Occam’s razor says: treat these as conscious.

• Self is agent that is the client of the mind-- namely the

reified interest of the genome.• Awareness is of semantic quantities: outputs of modules

affecting our decisions directly.“Comes together” at CDU?

• All thought is execution of code, including qualia.Tell me everything physical there is to tell about what isgoing on in a living brain and… you won’t have told me about the hurtfulness of pains, the itchiness of itches, pangs of jealousy or about the characteristic experience of tasting a lemon, smelling a rose, hearing a loud noise,... (F. C. Jackson)

But in fact these are easy to explain...

Page 26: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

What is Thought?

• Thought is execution of a program that exploits the compact structure of the world

• Our program built on evolved modules the learning of which is coded in DNA.

• Consciousness arises as coding of decision maker/learner

• Theory will make predictions for gene expression, imaging, psychophysics…

Page 28: Ebaum@fastmail.fm. Meaning is the computational exploitation of the underlying structure of the world, and mind is execution of an evolved program, mostly

How do you find a compact explanation?

• Hard optimization problem• Local search/evolution.