55
Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in Computer Science Simon D. Levy Computer Science Department Washington & Lee University Fourth Regional Wittgenstein Workshop 11 March 2012 Socrates Plato Aristotle Aquinas Descartes Locke Hume Kant Frege Wittgenstein Angry Birds Phone Sunday, March 11, 12

Wittgenstein's Computer - Washington and Lee …home.wlu.edu/~levys/presentations/WittgensteinsComputer.pdf · Wittgenstein's Computer Wittgenstein (Early and Late) and Modeling in

Embed Size (px)

Citation preview

Wittgenstein's ComputerWittgenstein (Early and Late) and Modeling in Computer Science

Simon D. LevyComputer Science DepartmentWashington & Lee University

Fourth Regional Wittgenstein Workshop11 March 2012

Socrates Plato Aristotle

Aquinas Descartes Locke

Hume Kant Frege

Wittgenstein Angry Birds Phone

Sunday, March 11, 12

In Memoriam

Ruth Barcan Marcus 1921 - 2012

Sunday, March 11, 12

Origins: 1939

WITTGENSTEIN: I won’t say anything which anyone can dispute. Or if anyone does dispute it, I will let that point drop and pass on to say something else.

TURING: I understand but I don’t agree that it is simply a question of giving new meanings to words.

WITTGENSTEIN: Turing doesn’t object to anything I say. He agrees with every word.

TURING: I see your point.

WITTGENSTEIN: I don’t have a point.

[Diamond 1976]Sunday, March 11, 12

Computing: 1946

Sunday, March 11, 12

Computing: "2004"

Sunday, March 11, 12

Computing: 2010

Sunday, March 11, 12

Hierarchy

Sunday, March 11, 12

Hierarchy

Sunday, March 11, 12

Localist Representations + Explicit Rules =

Brittle Computations

cats

chase

dogs

AGENT PATIENT

Sunday, March 11, 12

Localist Representations + Explicit Rules =

Brittle Computations

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sequential pointer-following:Who chases cats?

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &6

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

One small error = total failure

cats

chase

dogs

AGENT PATIENT

Address Contents

0000 <chase>

0001 &12

0010 &8

0011 &7

0100 <dogs>

0101 <cats>

0110 <AGENT>

0111 &4

1000 <PATIENT>

1001 &5

Sunday, March 11, 12

Usage

The confusions which occupy us arise when language is like an engine idling, not when it is doing work.

Sunday, March 11, 12

Usage

Sunday, March 11, 12

Rule-Following

All this, however, can only appear in the right light when one has attained greater clarity about the concepts of understanding, meaning, and thinking. For it will then also become clear what can lead us (and did lead me) to think that if anyone utters a sentence and means or understands it he is operating a calculus according to definite rules. 

Sunday, March 11, 12

Family Resemblances

I can think of no better expression to characterize these similarities than "family resemblances"; for the various resemblances between members of a family: build, features, colour of eyes, gait, temperament, etc. etc. overlap and cries-cross in the same way.-And I shall say: 'games' form a family.

Sunday, March 11, 12

Critique of Pure Reductionism

Suppose everyone had a box with something in it: we call it a "beetle". No one can look into anyone else's box, and everyone says he knows what a beetle is only by looking at his beetle. — Here it would be quite possible for everyone to have something different in his box.

Sunday, March 11, 12

Blue and Brown Books

Sunday, March 11, 12

WALK

WALKED

Sunday, March 11, 12

ROAR

ROARED

Sunday, March 11, 12

SPEAK

SPOKE

Sunday, March 11, 12

GO

WENT

Sunday, March 11, 12

ignores(mary, john)

Mary won’t give John the time of day.

Sunday, March 11, 12

Challenges (Jackendoff 2002)

Sunday, March 11, 12

I. The Binding Problem

+

? ? ? ?Sunday, March 11, 12

II. The Problem of Two

+

? ? ?Sunday, March 11, 12

III. The Problem of Variables

ignores(X, Y)

X won’t give Y the time of day.

Sunday, March 11, 12

Vector Symbolic Architectures

(Plate 1991; Kanerva 1994; Gayler 1998)

Sunday, March 11, 12

Tensor Product Binding(Smolensky 1990)

Sunday, March 11, 12

Binding

Sunday, March 11, 12

Bundling

+ =

Sunday, March 11, 12

Unbinding (query)

Sunday, March 11, 12

Lossy

Sunday, March 11, 12

Lossy

Sunday, March 11, 12

Cleanup

Hebbian / Hopfield /

Attractor Net

Sunday, March 11, 12

Reduction(Holographic; Plate 2003)

Sunday, March 11, 12

Reduction(Binary;

Kanerva 1994,Gayler 1998)

Sunday, March 11, 12

Composition / Recursion

Sunday, March 11, 12

Variables

X

john

Sunday, March 11, 12

Family Resemblances

• Both "primitive" concepts (cat, dog, chase) and "complex" ones (dogs chase cats unless the cats are bigger than them, etc.) are represented in the same way (N-dimensional vector)

• So we can use the same metric (e.g. Euclidean distance) to describe resemblances between objects of arbitrary complexity.

Sunday, March 11, 12

Rule-Following without Rules

• VSA can generalize ("learn a rule") from a single exemplar, because anything can be a "variable" (c.f. Moser 2012)

• E.g. What is the dollar of Mexico?

Sunday, March 11, 12

“What is the Dollar of Mexico?” (Kanerva 2009)• Let X = <country>, Y = <currency>,

A = <USA>, B = <Mexico>

• Then A = X*U + Y*D, B = X*M + Y*PD*A*B =

D*(X*U + Y*D) * (X*M + Y*P) =

(D*X*U + D*Y*D) * (X*M + Y*P) =

(D*X*U + Y) * (X*M + Y*P) =

D*X*U*X*M + D*X*U*Y*P + Y*X*M + Y*Y*P =

P + noise

Sunday, March 11, 12

The Random Beetle in the Box

• VSA representations are essentially random.

• So any two individual agents' internal representations of the same concepts will be unrelated to each other, though perfectly functional.

• VSA reps can also be expressed as neural spike trains (Stewart / Bekolay Eliasmith 2011), making VSA a plausible candidate for perceptual grounding of symbols (à la Barsalou 1999 ).

Sunday, March 11, 12

SourcesBarsalou, L.W. (1999) Perceptual symbols systems. Behavioral and Brain Sciences 22.

Diamond, C., ed. (1976) Wittgenstein's Lectures on the Foundations of Mathematics, Cambridge, 1939, Hassocks: Harvester Press.

Dror, I. E. and M. Dascal (1997) Can Wittgenstein help free the mind from rules? The philosophical foundations of connectionism. In D. Johnson and C. Erneling (eds.), The Future of the Cognitive Revolution. Oxford University Press.

Kanerva, P. (2009) Hyperdimensional computing: an introduction to computing in distributed representation with high-dimensional random vectors. Cognitive Computation 1(2).

Klagge, J.C. (1989) Wittgenstein and neuroscience. Synthese 78.

Moser, C. (2012) Schmagency and the formal character of constitutive principles. Presented at the Fourth Regional Wittgenstein Workshop (10 March 2012), Lexington, Virginia.

Stern, D.G. (1991) Models of memory: Wittgenstein and cognitive science. Philosophical Psychology 4 (2).T

Stewart, T., T. Bekolay, and C. Eliasmith (2011) Neural representations of compositional structures: representing and manipulating vector spaces with spiking neurons. Connection Science 23 (2).

Wittgenstein, L. (1921/1975) Tractatus Logico-Philosophicus (trans. D.F. Pears and B.F. McGuiness). Routledge.

Wittgenstein, L. (1933/1965) The Blue and Brown Books. Harper Torchbooks.

Wittgenstein, L. (1953/1973) Philosophical Investigations (3rd Edition, trans. G.E.M. Anscombe). Prentice Hall.

Sunday, March 11, 12