Machine Consciousness and Cognitive Robotics

Two Research Agendas or One?

Murray ShanahanImperial College London

Dept. Electrical Engineering

2

Overview

ConsciousnessCognitive Robotics

Motivation and methodologyLudwig: a humanoid robotPerception as abduction

Philosophical Implications

3

Expect Changing of HatsEngineer Philosopher

4

As a Philosopher…

5

“Consciousness”The word “consciousness” seems to conflate several concepts. To be conscious is

To be aware of the physical and social worldTo be aware of the selfTo have an inner world & an internal monologueTo offer an emotional response to life

If we have a theory that accounts for these things, will the philosophical puzzle that it is “like something” to be a conscious creature disappear?Such an account will at the same time be an explanation of how the brain generates intelligent behaviourThe engineer’s considerations may be pertinent

6

A Blueprint for the Human Mind

Action

EmotionWORLD

Imagination

Perception

7

The Imagination is the KeyThe Imagination is a multi-media, virtual reality “theatre”, where stories about the body and the self are played out. These stories

are influenced by the present situation according to Perception,elicit a secondary response from Emotion,and thereby influence the decisions taken by Action

The Imagination rehearses trajectories through an abstraction of sensory-motor space.The inner loop of the Imagination mirrors the outer Action-World-Perception loopThe stories include visual images, sounds and plenty of conversation (the inner monologue)

8

Seeing AsPerceiving an object (or situation or event) entails seeing it as a member of one or more categories

Seeing a collection of edges and surfaces as a rock or a deer

To see X as a Y is to see X as a potential participant in a whole repertoire of little stories in the Imagination involving Ys

Hunting a deer, cooking its meat, using its antlers

The power of the human imagination lies in its talent for projecting things onto novel categories

Seeing a horse as a means of transportSeeing a rock as a hammer

9

Little Spatial and Social Stories

All these stories are grounded in small infantile sensory-motor episodes (cf: Lakoff)

Putting things in containers / taking things out of containersFollowing a pathGiving things to others / taking things from others

Abstract concepts built on these concrete foundationsFor example, experiences relating to containment ground

Bank accountsVariables in programming languages

Abstract concept of Self built around core representation of the body

10

By the Way…A full theory of mind also needs to account for “altered states of consciousness”, such as those experienced during

DreamsHallucinationsMeditation

11

As an Engineer…

12

What Do I Want to Do?Build an artefact that has as near to human-level intelligence as possibleDo so in a principled way, by uncovering the fundamental laws that govern the design of such artefactsIf the results impact on psychology and/or philosophy, so much the betterBut ultimately the project will stand or fall as an engineering exerciseAlbeit engineering with a “philosophical edge”

13

Motivating Conjecture 1Abstract Concepts are Grounded in

Sensory-Motor Interaction with a World of Spatio-Temporally Located Objects

Consider the order in which locomotion, communication, manipulation, and language appear in evolutionary historyConsider their order of appearance in child developmentClassical AI hasn’t delivered using top-down methodologySymbols have to be grounded to have meaning“The core of our conceptual systems is directly grounded in perception, body movement, and experience” (G.Lakoff)

14

Motivating Conjecture 2The Ability to Handle Uncertainty and Incompleteness Is Central to Human-

Level CognitionUncertainty is an unavoidable consequence of the limitations of any sensorIncompleteness is an inevitable consequence of having a point-of-view, which is in turn a consequence of having a spatial location

Can’t see back and front of an object at the same timeIn cluttered environment, occlusion is the normMost of the world is too distant to be known

15

Motivating Conjecture 3Perception, Action and Cognition Form an

Inseparable TrinityCognition is constrained by need for reactivity

Perception is intractable without help of cognition

Behaviour-based approaches are limited by lack of cognition

Evolutionary pressure behind development of cognition was the benefit of a better closed-loop control system

Fragmentation of AI spawned irreconcilable sub-disciplines that lost sight of the big challenge

16

Two Design PrinciplesIntelligent behaviour will result from correct reasoning with

correct representation

Adaptive behaviour is the product of closed-loop control

These align with the approaches of classical AI and biologically-inspired robotics (cybernetics)To reconcile these insights, explicit representation and reasoning are inserted in a fast feedback loop

Use of explicit reasoning must not entail the deployment of computationally unbounded processesUse of explicit representation must not entail the construction of a complete and accurate model of the world

17

Methodological ConclusionIf we want to engineer an artefact with human-level cognitive skills, central topics include

Representing and reasoning about shape and spaceDealing with incompleteness and uncertainty

So let’s build a robot that perceives, reasons about, and acts upon objects having a variety of different shapes in a messy environmentThe architecture should integrate explicit representation and explicit reasoning with closed-loop control

18

The Present ChallengeBuild a humanoid robotFill its workspace with everyday objectsProgram it to attend to interesting objects given visual cuesMake it construct representations of objects from their appearanceLet it nudge interesting objectsMake the quality of its representations improve with the resulting new information

19

Ah! Ludwig

“What am I supposed to do with these things? All I know about is first-order predicate calculus!”

20

Ludwig: A Humanoid Robot

Pan-and-tilt headStereo visionTwo articulated armsThree degrees-of-freedom eachUpper torso only

21

LUDWIG Nudging a Ball

QuickTime™ and a decompressor

are needed to see this picture.

QuickTime™ and a decompressor

are needed to see this picture.

This is a simple demonstration of the basic mechanical, electrical, and control system working together

22

A View of Robot PerceptionThe role of robot perception is to transform raw sensor data into a “correct” representation of the external worldThe idea of correctness makes sense if the representations are symbolic and have a compositional semanticsTo engineer our robot in a principled way, we need a formal account of how this is doneThis account will explain how symbols and sentences denote, how they are grounded, how they refer, how they acquire intentionalityFinally we need an implementation that conforms to the theory

23

Perception as Abduction

describes impact of world on sensors describes object shapes & locations

Must be consistent & comprise abducible formulaeNeed method for handling multiple s

describes data from cameras, sonar, etc

€

∧|=Background

theoryObjects

in the world

Sensor data

24

The Character of comprises first-order predicate calculus formulae describing

Generic laws about actions and changeExpressed in a formalism for describing actions, such as the event calculusMust incorporate a solution to the frame problem

Generic laws about spatial occupancy, solidity, persistence, occlusion, etcThe effects of specific actions

If you move forwards your location changes in a certain wayIf you rotate an object, its aspect changes in a certain way

25

Desiderata for a Full Theory of Perception

An agent’s perceptual system should be tuned to those aspects of the world that are pertinent to its needs (J. von Uexküll’s Umwelten)To perceive something should be to perceive its use (J. J. Gibson’s affordances)Perception should be active: gathering new sensor data facilitates the interpretation of old sensor dataAn attention mechanism should be used to manage the volume of sensor data while ensuring its utilityThe flow of information between perception and cognition should be bidirectional

26

The Visual ImaginationSymbolic, sentence-like representations are only half the story

Good for abstractionGood for handling incompleteness and uncertainty

Topographic, image-like representations offer complementary features

Spatial properties come for free (shape, orientation, etc)Animation can be used to simulate effects of action and change

Combining symbolic and topographic representationsPixel arraysAnimatedMultiple levels of granularityRegions of array labelled with abductively determined categories

27

As a Philosopher…

28

Engineering with a Philosophical Edge

“Much of what we think will of course prove to be mistaken; that is one advantage of real

experiments over thought experiments.”(Dennett on the Cog project)

29

Philosophical ParallelsThe Philosopher and the Engineer

Intentionality

Propositional attitudes

Kantian idealism

Consciousness

30

IntentionalityThe Philosopher

“How is intentionality possible? How is it that states

of mind can be about the world? How are symbols

grounded?”

The Engineer

“A clear and well-defined causal story can be told about how sentences in

Ludwig’s memory come to be there. The story starts with the state of the world, moves on to how this causes

electrical signals in the robot’s sensors, and concludes with computational processes that transform these into

representations. The abductive account spells out exactly the sense in which the

resulting representations are correct.”

31

Propositional AttitudesThe Philosopher

“What are propositional attitiudes? What are beliefs, desires, and intentions, and what part do they play in a

(functionalist) theory of mind?”

The Engineer

“A precise causal story can be told about how representations in Ludwig’s memory arrive there, interact with each other, and issue in robot actions. The

story involves well-defined computational processes of sensor data

assimilation and planning through logical inference.

(You don’t seriously think the human mind is like this do you?)”

32

Kantian IdealismThe Philosopher

“The natural world as we know it ... is thoroughly conditioned by [certain]

features: our experience is essentially experience of a

spatio-temporal world of law-governed objects conceived of as distinct from our temporally

successive experiences of them” (Strawson on Kant)

The Engineer

“Ludwig’s background theory must contain axioms about time, action, change, space, and shape,

and the hypotheses that explain sensor data are

therefore constrained to be expressed in those terms.”

33

ConsciousnessThe Philosopher

“But what is the nature of consciousness? How does it

arise in physical matter?”

The Engineer

“What? Hmmm, well, let me take a step back…”

34

Conclusion

Machine consciousness and cognitive robotics: two research agendas or one?

Two agendas with a very large area of overlap

35

And Finally ...

The Philosopher

“Where’s the nearest pub?”

The Engineer

“Where’s the nearest pub?”