cse@buffal

o

The GLAIR Architecturefor Cognitive Robotics

Stuart C. ShapiroDepartment of Computer Science & Engineering

and Center for Cognitive ScienceState University of New York at Buffalo

shapiro@buffalo.edu

cse@buffal

o

S. C. Shapiro2

GLAIR:Grounded Layered Architecture

with Integrated ReasoningFocus of This Talk:

ConnectingReasoning Mind

withActing/Sensing Body

CogRob2010

cse@buffal

o

3CogRob2010 S. C. Shapiro

Motivations• Add acting and sensing to a reasoning agent.

– First person reasoning; on-line acting & sensing.• Layers

– Motivated by mind/body connections/distinctions.– Let same mind be plugged into different bodies.

• Embodiment– Origin of beliefs in sensation & proprioception.– First-person privileged knowledge of own body.

• Situatedness– Has a sense of where it is in the world.

• Symbol grounding– In body-layer structures.– Symbol as pivot between various modalities.

cse@buffal

o

S. C. Shapiro4

Outline

The Layers

Symbol Grounding

Putting it Together: Two examples

CogRob2010

cse@buffal

o

Cog Rob 2010 S. C. Shapiro 5

KL

PMLa

PMLb

PMLc

SAL

Mind

BodyIndependentof lower-body

implementation

Hearing

Vision

Motion

Speech WORLD

I/P s o c k e t s

GLAIR Architecture

Dependenton lower-bodyimplementation

Proprioception

cse@buffal

o

S. C. Shapiro

Sensori-Actuator Layer

• Sensor and effector controllers

CogRob2010 6

cse@buffal

o

S. C. Shapiro7

Perceptuo-Motor Layer

• PMLa• PMLb• PMLc

CogRob2010

cse@buffal

o

S. C. Shapiro

PMLc

• Abstracts sensors & effectors• Body’s behavioral repertoire

CogRob2010 8

cse@buffal

o

S. C. Shapiro

PMLb

• Translation & Communication– Between PMLa & PMLc

• Highest layer that knows body implementation

CogRob2010 9

cse@buffal

o

S. C. Shapiro

PMLa• Grounds KL symbols

– Perceptual structures– Implementation of primitive actions

• Registers for Embodiment & Situatedness– Deictic Registers– Modality Registers

CogRob2010 10

cse@buffal

o

The Knowledge Layer• Implemented in SNePS• Agent’s Beliefs• Representations of conceived of entities• Semantic Memory• Episodic Memory• Quantified & conditional beliefs• Plans for non-primitive acts• Plans to achieve goals• Beliefs re. preconditions & effects of acts• Policies: Conditions for performing acts• Self-knowledge• Meta-knowledge

CogRob2010 S. C. Shapiro 11

cse@buffal

o

S. C. Shapiro12

SNePS

• A KRR system• Every non-atomic expression is simultaneously

– An expression of SNePS logic– An assertional frame– A propositional graph

• Every SNePS expression is a term– Denoting a mental entity

CogRob2010

cse@buffal

o

S. C. Shapiro13

Ontology of Mental Entities• Entity

– PropositionAgent can believe it or its negationIncludes quantified & conditional beliefs

– ActAgent can perform it

– PolicyCondition-act rule agent can adopt

– ThingOther entities: individuals, properties, times, etc.

CogRob2010

cse@buffal

o

S. C. Shapiro14

Outline

The Layers

Symbol Grounding

Putting it Together: Two examples

CogRob2010

cse@buffal

o

S. C. Shapiro

Entities, Terms, Symbols, Objects

• Agent’s mental entity: a person named Stu

• SNePS term: b4

• Object in world:

15CogRob2010

cse@buffal

o

S. C. Shapiro

AlignmentMind (KL)

Body (PML/SAL)

World

KL term

PML structure

Object/Phenomenon Action

CogRob2010 16

cse@buffal

o

World Objectsto Feature Tuples

<Height, Width, Texture, . . . >

WorldPML/SAL

17S. C. ShapiroCogRob2010

cse@buffal

o

S. C. Shapiro18

Feature Tuplesto KL Terms

<Height, Width, Texture, . . . >

PML/SALKL

ProperName(b4, Stu)

Alignment

CogRob2010

cse@buffal

o

S. C. Shapiro19

Incomplete PML-Descriptions

<Height, nil, nil, . . . >

PML/SALKL

Height(b4, b12)

CogRob2010

cse@buffal

o

S. C. Shapiro20

Unifying PML-Descriptions

<x, nil, nil, ... >

PML/SALKL

b20

b30

b31

b6

<nil, y, nil, ... >

<nil, nil, z, ... >

<x, y, z, ... >

Isa

Prop

Prop

CogRob2010

cse@buffal

o

S. C. Shapiro21

Deictic Registers

For being situated in the worldPML registers hold KL terms

I term denoting agentYOU term denoting dialogue partnerNOW term denoting current time

CogRob2010

cse@buffal

o

S. C. Shapiro22

Modality Registers

For privileged first-person knowledge of what agent is doing

Register for each modality, m, holdsKL term denoting act that m is engaged in

CogRob2010

cse@buffal

o

S. C. Shapiro23

Outline

The Layers

Symbol Grounding

Putting it Together: Two examples

CogRob2010

cse@buffal

o

S. C. Shapiro24

<green, nil>

PML/SALKL

b20

b30

b6

<nil, robotic>

Isa

Prop

CogRob2010

m2

Find a green robot.

(find )

cassie

m75

m76

robbie

<green, robotic>

VISION

Language-Mind-World-Mind

WORLD

cse@buffal

o

Building Episodic Memory

CogRob2010 S. C. Shapiro 25

KL

PML

e1

I

a1 b1

m1!

t1 e2

a2

m2!

t2

m3!

NOWCOUNT n

hom0

qm4!

before afterevent eventtime

time

time

act act

agent

agent

duration

ACT

cse@buffal

o

S. C. Shapiro26

Summary

• GLAIR is an architecture:– To connect mind and body

• For symbol grounding– To provide first-person privileged knowledge of

• The body• What it’s doing

– To provide situatedness in the world• Via deictic registers

CogRob2010

cse@buffal

o

S. C. ShapiroCSE 71927

Collaborators

Past and present members ofSNeRG: The SNePS Research Grouphttp://www.cse.buffalo.edu/sneps/